wissel.net

Usability - Productivity - Business - The web - Singapore & Twins

NotesSessions, XPages and Threads


When you build an XPages application, long running operations will time out. After all there is a timing limit how long a browser (or XPiNC) client (and the user) will wait for a response. You could resort to launching an agent via a console command, but besides the security consideration it is quite a hack and headache. Mixing XPages and agents doesn't have a happy ending (and should be confined to the upgrading phase of your classic application).
Enter Java multi-threading. I call it " Teenage-mode": your code does multiple things at the same time and doesn't seem to get distracted.
When you read the classics you will find threading hard and rather threatening (pun intended), but thanks to progress in Java6 and above it isn't too hard.
There is the Executor framework that handles all your Runnables. There are 2 challenges to overcome: one specific to Notes and one general to all concurrency programming. The former: The Notes Session class is not threadsave and can't be shared between threads, the later: you have to get your queueing right:

(Image courtesy of Lee Chee Chew)
To get a new Session object, we have the session cloner, thanks to the extension library. All your threads implement the Runnable interface that is later called by the executor framework. There are 2 approaches for background task: one where you expect the thread to return some value at some time and the other where a thread runs its course and terminates silently when done. For a current project I opted for the later. The threads contain a callback class where they report back what they have to say. To make it all work we need:
  1. A management class that executes the threads for us. Preferably long living (like session or application beans)
  2. A cloned Notes Session
  3. One or more classes implementing the runnable interface
Let's look at some code, first the class that runs in the background:
package com.notessensei.demo ;

import java.util.Collection ;
import java.util.Map ;
import lotus.domino.Database ;
import lotus.domino.NotesException ;
import lotus.domino.Session ;
import com.ibm.domino.xsp.module.nsf.NSFComponentModule ;
import com.ibm.domino.xsp.module.nsf.NotesContext ;
import com.ibm.domino.xsp.module.nsf.SessionCloner ;

public abstract class AbstractNotesBackgroundTask implements Runnable {
    protected final Session                     notesSession ;
    protected final Collection < String >          callBackMessages ;
    private SessionCloner                   sessionCloner ;
    private NSFComponentModule              module ;
   
    public AbstractNotesBackgroundTask ( final Session optionalSession, final Collection < String > messageHandler ) {
        this. callBackMessages = messageHandler ;
        // optionalSession MUST be NULL when this should run in a thread, contain a session when
        // the class is running in the same thread as it was constructed
        this. notesSession = optionalSession ;
        this. setDominoContextCloner ( ) ;
    }

    public void run ( ) {
        this. callBackMessages. add ( "Background task run starting: " + this. getClass ( ). toString ( ) ) ;
        try {
            Session session ;
            if ( this. notesSession == null ) {
                NotesContext context = new NotesContext ( this. module ) ;
                NotesContext. initThread (context ) ;
                session = this. sessionCloner. getSession ( ) ;
            } else {
                // We run in an established session
                session = this. notesSession ;
            }
           
            /* Do the work here */
            this. runNotes ( ) ;
           
        } catch ( Throwable e ) {
            e. printStacktrace ( ) ;
        } finally {
            if ( this. notesSession == null ) {
                NotesContext. termThread ( ) ;
                try {
                    this. sessionCloner. recycle ( ) ;
                } catch (NotesException e1 ) {
                    e1. printStacktrace ( ) ;
                }
            }
        }
        this. callBackMessages. add ( "Background task run completed: " + this. getClass ( ). toString ( ) ) ;
    }

    private void setDominoContextCloner ( ) {
        // Domino stuff to be able to get a cloned session
        if ( this. notesSession == null ) {
            try {
                this. module = NotesContext. getCurrent ( ). getModule ( ) ;
                this. sessionCloner = SessionCloner. getSessionCloner ( ) ;
            } catch ( Exception e ) {
                e. printStacktrace ( ) ;
            }
        }
    }
   
    protected abstract void runNotes ( ) ;
}
You only need to subclass it and implement runNotes() to get started. However you probably want to hand over more parameters, so you need to modify the constructor accordingly. Please note: you can't bring Notes objects across thread boundaries. Next the class that controls all the threads. It should live in the session or application.The class is actually quite simple:
package com.notessensei.demo ;

import java.io.Serializable ;
import java.util.concurrent.ExecutorService ;
import java.util.concurrent.Executors ;

public class NotesBackgroundManager implements Serializable {
    private static final long               serialVersionUID        = 1L ;
    // # of threads running concurrently
    private static final int                THREADPOOLSIZE          = 10 ;
    private final ExecutorService           service                 = Executors. newFixedThreadPool (THREADPOOLSIZE ) ;

    public CSIApplication ( ) {
        // For use in managed beans
    }

    public void submitService ( final Runnable taskDef ) {
        if (taskDef == null ) {
            System. err. println ( "submitService: NULL callable submitted to submitService" ) ;
            return ;
        }
        // Execute runs without return
        this. service. execute (taskDef ) ;
    }

    @ Override
    protected void finalize ( ) throws Throwable {
        if ( ( this. service != null ) && ! this. service. isTerminated ( ) ) {
            this. service. shutdown ( ) ;
        }
        super. finalize ( ) ;
    }
}
If you rather want the thread returning something at the end of the run (which you must poll from the ExecutorService, you would call ExecutorService.submit and get back a Future holding the future result, but that's another story for another time.
The final missing piece is to define NotesBackgroundManager as managed bean e.g. background and you can launch a background task in SSJS: background.submitService(new com.notessensei.demo.SampleNotesBackgroundTask());
As usual YMMV.

Posted by on 23 July 2013 | Comments (7) | categories: XPages

Comments

  1. posted by Nathan T. Freeman on Tuesday 23 July 2013 AD:
    Wonderful article. Concurrency is something that Domino developers really should learn more about.

    "...The Notes Session class is not threadsave(sic) and can't be shared between threads..."

    We're all friends here, so I'm going to be a little pedantic, because I used to think this too.

    It's not that you can't share a Session (or a Database or a Collection or whatever) between threads; it's that Domino objects *must* be recycled on the same thread in which they were established. So if you have Thread 1, and you have a Session and a Database that you then give to Thread 2, which creates Views, DocumentCollections, Documents and Items from there, all those View, DCs, Docs and Items must be recycled on Thread 2, but the Database and the Session MUST NOT be recycled on Thread 2. They can only be recycled on Thread 1 and obviously only after Thread 2 finishes its work. (Technically you *can* recycle on a different thread, if you don't mind the side-effect of crashing your client or server.)

    What deepens the challenge still further is the following: Let's say you have Thread 1 with a Session, and you create Threads 2 and 3 with the same Session. Thread 2 gets a handle on foo.nsf and begins some processing. Thread 3 is processing something else, but also needs a handle on foo.nsf a few milliseconds later. Now you have a problem -- 2 and 3, because they share a Session, actually get the same handle on foo.nsf. But you don't really have any way of knowing that.

    Recycle in 3, and you crash. Recycle in 2, and if 3 is still working, it gets the dreaded "Object has been removed or recycled" error.

    You can try to keep track of who got what objects first, and who else accessed them and whether they still need them. And then you can .join all your Threads so they don't release any concurrently accessed objects until everyone else is finished with them. But you have to do all this with a lot of synchronized Maps and it's a serious performance killer. At some point, you ask yourself: why am I bothering with multiple threads when I have to synchronize everything that's happening anyway?

    Or at least, that's what I did when trying to solve the problem for the OpenNTF Domino API. Once I got a mostly workable solution done, I ran some benchmarks, found that it was about 8 times slower, and said "this is dumb. Just don't share Sessions between threads."
  2. posted by Stephan Wissel on Tuesday 12 November 2013 AD:
    To keep my simple mind from blowing up, I invented the simple rule: if you make them, you dispose them. So if a method defines a NotesObject that object must be removed in the same method. Works most of the time. Emoticon biggrin.gif stw
  3. posted by Greg on Friday 29 November 2013 AD:
    Thanks a lot for this article, it is something that I sorely need! (I don't like making a user wait while the application goes off on its own little tangent (task).

    I was playing around with this because I want to create multiple documents while dynamically calculating and fetching configuration documents from other databases etc. And I keep getting the following error in the server log:

    29.11.2013 11:14:17 HTTP JVM: Exception in thread "pool-4-thread-1"
    29.11.2013 11:14:17 HTTP JVM: java.lang.IllegalStateException: NotesContext not initialized for the thread
    29.11.2013 11:14:17 HTTP JVM: at com.ibm.domino.xsp.module.nsf.NotesContext.getCurrent(NotesContext.java:123)
    29.11.2013 11:14:17 HTTP JVM: at com.ibm.domino.xsp.module.nsf.ModuleClassLoader$DynamicClassLoader.loadClass(ModuleClassLoader.java:383)
    29.11.2013 11:14:17 HTTP JVM: at java.lang.ClassLoader.loadClass(ClassLoader.java:638)
    29.11.2013 11:14:17 HTTP JVM: at de.holistic.utils.multithreading.AbstractBackgroundTask.run(AbstractBackgroundTask.java:29)
    29.11.2013 11:14:17 HTTP JVM: at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:906)
    29.11.2013 11:14:17 HTTP JVM: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:929)
    29.11.2013 11:14:17 HTTP JVM: at java.lang.Thread.run(Thread.java:738)

    (sorry if that is a bit difficult to work with)

    I am using Domino 901, Notes 900, and the latest 900 ExtLibs. Do you have any idea why I might be getting the IllegalStateException / NotesContext not initialized for the thread message? I am guessing that my first step should be to get my versions consistent with each other and try again, but I wanted to ask for your input as well.
  4. posted by Greg on Friday 29 November 2013 AD:
    to make it easier: [link= stackoverflow.com/questions/20285907/xpage-concurrency-notescontext-not-initialized] here is a stack overflow question [/link]
  5. posted by Panu Haaramo on Monday 13 October 2014 AD:
    I'm getting the same exception as Greg, NotesContext.getCurrent() fails.

    Any idea why the same code works for you but not for us? Thanks.
  6. posted by Stephan H. Wissel on Monday 13 October 2014 AD:
    I haven't revisited the code for a rather long time. The smartest way is to move on and use the OpenNTF Domino API. Nathan did an outstanding job to make that even easier.
  7. posted by Panu Haaramo on Tuesday 14 October 2014 AD:
    Thanks, haven't tried that yet. Actually your code works. I was not getting module and SessionCloner in the constructor.