wissel.net

Usability - Productivity - Business - The web - Singapore & Twins

By Date: September 2012

Starting Domino on Linux using UPSTART


When running Domino on a proper platform (AIX, Solaris, Linux) starting and stopping the Domino server was left to customizing a script from a technote or a Redbook's FTP site, as far as official IBM resouces go. Of course the professional source is Daniel Nashed's ultimate Domino startup script. One script to rule them all.
On Linux however the way services are started has changed a while ago. The prefered method (definitely in Ubuntu, but also Fedora, RedHat and OpenSuse as option) is Upstart (there seems to be a push towards systemd, but that's a story for another time).
Upstart allows for a more flexible control and faster boot times of your environment. To configure your Domino on Linux we will use 2 scripts and one configuration file for each instance (inspired by the same approach for node.js).
The first file is /etc/init/domino.conf with the following content:
# Sample job script for domino, experimental - use at your own risk, don't use in production
description 'lotus domino upstart script'
author '@notessensei'

#Stop on shutdown - but no start directive - since it gets started by another script
stop on shutdown

#Instance allows for multiple scripts running
instance "Domino - $NAME"

# Restart if it was crashing, with a limit
respawn
respawn limit 5 60

# Will go into the background
expect fork

# Kill timeout 20 sec - to give Domino a shutdown chance
kill timeout 20

# Check for the password file
pre-start script
    . /etc /domino / $NAME.conf
    # Ensure the pwd file is there and has the right owner/access
    if [ ! -f $PWD_LOCATION ]; then
        touch PWD_LOCATION
    fi
    chmod 0400 $PWD_LOCATION
    chown $SERVER_USER: $SERVER_GROUP $PWD_LOCATION
end script

# The script to start the server
script
    . /etc /domino / $NAME.conf
    exec sudo -u $SERVER_USER -c "cd ${DATA_LOCATION}; cat ${PWD_LOCATION}|PWD_LOCATION/server" >> $LOG_TO 2 >& 1 &
end script

# Run before shutdown - tell Domino to go down peacefully
pre-stop script
     . /etc /domino / $NAME.conf
    exec sudo -u $SERVER_USER -c "cd ${DATA_LOCATION}; /opt/ibm/lotus/bin/server -q"
end script

# Make sure it is really dead
post-stop script
    . /etc /domino / $NAME.conf
    exec sudo -u $SERVER_USER -c "cd ${DATA_LOCATION}; /opt/ibm/lotus/bin/nsd -kill"
end script
Secondly you create the configuration file in /etc/domino/server1.conf (you need to create the directory if needed, it isn't there by default):
#Configuration variables for Domino instance startup
#User and group for Domino
SERVER_USER= "domino"
SERVER_GROUP= "domino"
#Where does the data go
DATA_LOCATION= "/home/domino/server1/data"
#Must exist and have 0400 doe SERVER_USER:SERVER_GROUP
PWD_LOCATION= "${DATA LOCATION}/.domino.pwd"
#Log file
LOG_TO= "${DATA LOCATION}/domino.log"
The script will be able to start the domino instance using start domino NAME=server1. For additional instances ( partitioned servers) you only need to create an additional conf file in /etc/domino.
The final missing piece is the script that starts all the configured instances. Here we can more or less copy Ian's node script as /etc/init/alldomino.conf:
description 'Start all domino instances at boot'
author '@notessensei'

start on (local-filesystems and net-device-up )

task

script
  for file in ` ls /etc /domino /*.conf ` ; do
    filename= ` basename ${file%.*} `
    start domino NAME= $filename
  done
end script
That's all you need. As usual YMMV.

Posted by on 25 September 2012 | Comments (0) | categories: Linux Show-N-Tell Thursday

Creating Notes 8.5.3++ plug-ins with Eclipse 4.2


One skill that entitles you to the secret handshake is ability to develop plug-ins for the Lotus Notes clients. Sadly that is one of the technologies that held great promises (client side mashups anyone), that was clobbered by being to complicated, buggy and the rise of the "mobile first" mantra.
Still the Plug-in Jedi are with us and there are a number of wildly successful, useful and feature rich plug-ins that were contributed from outside IBM. Of course special mention goes to The Master's work.
One big stumbling block for development is the Expeditor Toolkit which is stuck on Eclipse 3.4.2 (while the current version of Eclipse is 4.2). Using the toolkit it is just a few clicks to create a runtime/debug configuration to test your plug-ins, without you are in for parameter guessing.
When searching for information you will find Mikkel's instructions for 3.5/8.5.2 and the entry in the Designer Help file (for the later one you need to know exactly what you are looking for). But both instructions won't work for Notes 8.5.3 ot 8.5.4. With the help of special friends I figured it out:
  • Install Eclipse 4.2 (Classic will do)
  • In Window - Preferences - Java - Installed JRE add the Notes JRE (I called mine Notes854)
  • In Window - Preferences - Plug-in Development - Target Platform add an new entry (based on an empty template) and add the directory location /opt/ibm/lotus/notes/framework/rcp/eclipse and /opt/ibm/lotus/notes/framework/shared/eclipse (adjust the directories to your path)
  • In Run - Run Configurations create a new Eclipse Application. Give it a name, in my example it is "SmartFile"
  • Leave the Workspace Data Location at its default ${workspace_loc}/../runtime-SmartFile
  • Run a product com.ibm.notes.branding.notes and point to the Runtime JRE you just configured (Notes854)
  • In Arguments (second tab) enter for the Program arguments (in one line):
    -personality com.ibm.rcp.platform.personality -console -pluginCustomization "${rcp_target}/../plugin_customization.ini"
  • in the VM argument enter the following. There seems to be no more need to specify variables or an installid
    -Declipse.registry.nulltoken=true
    -Dosgi.splashPath=platform:/base/../shared/eclipse/plugins/com.ibm.notes.branding
    -Djava.util.logging.config.class=com.ibm.rcp.core.internal.logger.boot.LoggerConfig
    -Dosgi.framework.extensions=com.ibm.rcp.core.logger.frameworkhook
    -Dcom.ibm.pvc.webcontainer.port=0
    -Djava.protocol.handler.pkgs=com.ibm.net.ssl.www.protocol
    -Dosgi.hook.configurators.exclude=org.eclipse.core.runtime.internal.adaptor.EclipseLogHook
    "-Xbootclasspath/a:${rcp_base}/rcpbootcp.jar"
    "-Djava.security.properties=file:${rcp_base}/rcp.security.properties" -Xss512K "-Drcp.data=${rcp_data}" "-Drcp.home=${rcp_target}/../.."

    (I don't know why some of the arguments are in quotes). Keep the working directory as default
  • In Plug-ins (3rd Tab): check all Target Platform plug-ins as well as your shiny new ones
  • In Configuration (4th tab) leave the "Use default location" On my machine:
    ${workspace_loc}/.metadata/.plugins/org.eclipse.pde.core/SmartFile
    and "Use an existing config.ini": ${rcp_base}/config.ini
  • I didn't touch Tabs 5-7
  • On my machine the Notes executable is on the path, but I don't know if that is mandatory
That's it. While you are on it, take a little refresher (keep in mind your target platform is still Eclipse 3.4.2)
As usual YMMV

Posted by on 18 September 2012 | Comments (2) | categories: Show-N-Tell Thursday

Calendars worked better when they were manual, did they?


Before calendars became electronic, having the right system in place was a signal of professionalism (admittingly abused as status symbols quite often) and calendars were very personal.
At the right level one had access to a personal assistant (the one without the D between the P and the A) who organized and maintained all aspects captured in the calendar. Inquiry of 3rd parties into your calendar was facilitated by a high powered neural network (a.k.a. the human brain) that translated the individual calendar entries into the information density deemed fit for the inquirer. " Information density" also called " information precision" is an interesting concept, that seems hard to translate into time planning software.
The information density decreases with distance to you
Your PA would know how to answer an inquiry about availability depending on your whereabouts, previous commitments and most importantly the relation/distance of the enquirer. It could range from a simple "No, try again another time" to "He's in Beijing, back next week" to "I'll slot a phone conference in for you at 17:00 GMT+8". With the rise of the digital assistants and calendars this flexible response got lost.
The first generation was entirely personal, while contemporary system will give you "available slots" or (if granted) a full detailed view. They still don't tell you where the other person is (you could use Google Latitude, Foursquare etc. for that) or will be (TripIt might be able to tell you).
Since calendars are no longer accessed only by a single person a conflict arises: one one hand we like it simple, on the other hand a lot of contextual information is needed to provide automated access at the right density level. Data protection and privacy concerns complicate matters further.
There are tons of solution attempts around which all fall short of taking information density into account. Some try to offer more than one calendar that you then can share with different people, some use tags, but I have yet to see one that can take an itinerary approach: I'm going on a trip to Orlando (usually in January). This sets timezone and location, but doesn't block time (unless a presence request indicates a different location outside a "reasonable radius"). Then as part of the trip I schedule sessions and meetings (that would block time then).
Short of having my own PA, that's what my calendar should be able to:
  • All the basic functions calendars have today: entries with and without people, repeating entries, reminders etc.
  • Hierarchical entries (the itinerary approach mentioned above)
  • Ability to switch into different timezones without altering the system timezone. Offer a shortcut based on where I am or will be in the day/week I'm looking at
  • Some clever mechanism to qualify entries, so enquiries (free time lookup etc) can return more or less information based on the enquirer (that one is really hard). Why can't a freetime lookup not include: "I need a specific location", "Online", "Phone" as qualifier. This includes what goes into my "public" calendar
  • A mechanism to figure out "What is the best option of the following given slots for the group of attendees" (probably online interactive)
  • The ability to track lead times (if I'm in the office and have a customer meeting at their place, I want the travel time blocked and eventually adjusted to traffic conditions)
  • The ability to plan preparation times when planning a meeting (that's a tricky one too) - so I can more efficiently plan time
  • Configurable meta data, so I can tie related calendar entries to customers, projects, goals etc.
  • Feature to drag task execution on and off the calendar - good for planning longer work (a task can have more than one calendar entry)
  • Ability to see public calendars on/off in my calendar in groups. Currently I need to switch them on/off one by one
  • more stuff I will think of, when working with the calendar again
Of course, your style would be completely different, so my wishlist wouldn't fit yours. Would it?

Posted by on 09 September 2012 | Comments (0) | categories: After hours Software

Mastery


An old Zen proverb states: " When the student is ready the master appears". It implies that it is a student's task to get ready. But it also implies that the master needs to be ready to teach.

In traditional craftsmanship the transition from journeyman to master is accompanied with the right and pledge to take on apprentices. So mastery must include teaching to lay the foundation for the next level: " poor is a pupil that does not surpass his master". After all it is " Purpose, Autonomy and Mastery" what motivates us.
Purpose Autonomy Mastery
Have an nice weekend!

Posted by on 08 September 2012 | Comments (0) | categories: After hours

The 2,147,483,648 NoteId limit


Every Note in a Lotus Notes database has 2 identifiers: a 32Byte Hex Universalid (UNID) and a NoteId (actually there are some more). The UNID is assigned once, never changes (unless you force it), is derived from a timestamp and stays unique across all replicas.
The 128Bit (incidentially the same size as IPv6) are divided in the first 64Bit from the replicaID and the second 64Bit for the individual documents (but you could overwrite that). Normal use gives you 2 64 possible values ( = 1.8 19 = 1,800 quintillion) for documents.
The Noteid on the other hand is unique only to one given database and changes when you create a new replica or run compact -c. For performance reasons (Notes is around for a while) this is a 4 byte Hex number (where only even numbers are used). For backward compatibility in the API that hasn't changed yet.
So you have 2 31 = 2,147,483,648 NoteIds at your disposal. A NoteId is assigned when a Note is saved in a database (could be design or data) and never reused even after a document is deleted and the resulting deleting stub expired and is removed.
When you have a very busy (large) database where many documents are constantly created and deleted you might hit that ceiling, even when the document count and database size seem moderate. The error condition is documented in Technote swg21220384 (the error can also occur in normal operation).
To my best knowledge you can't setup a DDM probe (or any other easy admin tool) to monitor " NoteId exhaustion". A suitable preventive remedy is to schedule regular compact -c for your busy databases.
Be careful with that, since the dose makes the poison and running that task every day will compound the side effects. If your database does need a daily compact, you have a structural problem in your application - come and see me. Typical intervals are weekly or monthly. Smart admins spread them out (so some compact is running every day). The side effect for compact -c on Windows (applied to SAN too) is disk fragmentation. So make sure you take care of that.

Posted by on 08 September 2012 | Comments (3) | categories: Show-N-Tell Thursday

XSS Vulnerabilities in Domino


An IBM Technote updated on 15 Aug 2012 points to a set of XSS vulnerabilities in the Lotus Domino server. You also can read the disclosure on seclist.org about that. But first go to your server configuration document and add:
DominoValidateFramesetSRC=1
Welcome back (You don't edit the notes.ini directly don't you!). When looking at XSS vulnerabilities, they follow the same pattern as SQL injections: Input that has been provided by the user is not fully sanitised and used for output generation. In web applications the " usual suspects" for such attacks are:
  • Framesets
  • URL parameters
  • Error and redirection pages
  • Form submissions
Finding all those is quite a task for both the developers and the attackers since URLs can be encoded in many different ways (@URLDecode is your friend). Luckily (for the first) and unfortunately (for the later) there is help available. Poking around in Domino made me add a new server rules ( Update thx to Sven to point that out) (not needed on Domino 8.5.4++):
  • Type of rule: HTTP response headers
  • Incoming URL pattern: */xsp/.ibmxspres/*
  • HTTP response codes: 404
  • Expires header: Don't add header
  • Custom header: Content-Type : text/plain (overwrite)
  • Type of rule: HTTP response headers
  • Incoming URL pattern: */xsp/.ibmmodres/*
  • HTTP response codes: 404
  • Expires header: Don't add header
  • Custom header: Content-Type : text/plain (overwrite)
Of course a server rule requires that you use the "Internet sites" configuration - since that configuration option was introduced in R6.0 it is high time you use them.
As usual YMMV

Posted by on 06 September 2012 | Comments (1) | categories: Show-N-Tell Thursday

Use Chrome web apps to access IBM Connections (and others)


Inside IBM (not counting the Greenhouse and SmartCloud) I'm a member of close to 200 communities. In some of them I'm quite active, in some I just follow the news stream. While the connections homepage keeps me updated with an activity stream, navigating to my favorite communities is uncomfortable.
I could add them to a browser bookmark, but that has no geek factor. Since the introduction ( read the full story) and reinvention of tiled start screens I grew fond of them and like to keep it that way. This is one of my Chrome start tabs (you can have many)
Chrome has tiles too
Luckily Google Chrome offers tiles too and with a few easy steps I can add my favorite destinations onto the start screen:
  1. Create a directory somewhere on your disk. Since you most likely will create more than one icon, put it under a common home. e.g. ~/MyGoogleIcons/wisselblog. Your "result will end in ~/MyGoogleIcons later on
  2. Now comes the hardest part: Create 2 GIF images for your application. One in the size 128x128px and one in 24x24px. If your community has an image, use that one, just make sure it still looks recognizable in the needed sizes. Save them to your directory
  3. Create a text file manifest.json in your directory. You can edit it with Notepad. The content should look like this (of course you put your own titles and URLs in there):
    {
      "name" : "NotesSensei Blog" ,
      "version" : "1" ,
      "icons" : { "24" : "wisselnet-24.gif" , "128" : "wisselnet-128.gif" } ,
      "app" : {
        "launch" : {
          "web_url" : "http://www.wissel.net/"
        }
      }
    }
    We are done with writing code here, if you want to know more check the full details what is possible
  4. Open Chrome, go to the extensions settings (Settings, Extensions) and check in the upper right corner "Developer mode". You will get two new buttons: "Load unpacked extension..." and "Pack extension..." If you just want to test your new icon, load is the right choice. When you are ready to finalize, so you can share, click pack
  5. You are prompted for the directory with your manifest.json file. Leave the key empty. When you update your extension, then you need to specify the key:
    Specify the home directory
    Everything went fine
  6. You are ready to install and to share. Drag the new crx file onto your settings page and Chrome will prompt you for install permission:
    Allow installation
    The new icon is on your page and you can drag to arrange it as desired. Friends and coworkers can use the crx file too
    It worked
  7. If you are an admin and want to roll-out the icons to a large user base, Chrome policies do that job for you
  8. Repeat for all the key communities
Nice little bonus: if your application is not available (e.g. you are not VPN connected to your intranet) you get a nice screen "not available" instead of a dumb 404.
As usual: YMMV

Posted by on 06 September 2012 | Comments (0) | categories: IBM IBM - Lotus