Poking around the iNotes HTTP API (Part 3) - retrieve message meta data

So far we learned about the URLs, how to retrieve the Outline that contains folders and views and how to utilize JavaScript to deal with an API tha hadn't been designed for language neutral access. This part will deal with the retrieval of messages. There are quite some formats, strategies and tweaks to consider:

• A message consists of headers (fields), and message parts that can be plain text, HTML, embedded images, attachments or other stuff (like the JSON used for embedded experiences). When to retrieve what part is an excellent question
• Depending on your use case, your final client might or might not have access to the original iNotes implementation (Firewall, Offline, VPN), so just showing the HTML and let it fetch missing pieces might not do the trick
• The HTML retrieved might not be well formed or contain links you don't want to execute (tracker images, tracker CSS, JavaScript stuff)
• Automatically retrieving the whole message can be network heavy if there are large images or attachments in the MIME parts, this needs to be planned carefully
• iNotes allows to retrieve the message fields as (kind-of) JSON using l_JSVars or retrieve the MIME headers using l_MailMessageHeader
• The content of the message is accessible via s_MailMemoReadBodyContent or UNID/Body?OpenField or l_MailMessageHeader&PresetFields=FullMessage;1 (as raw Mime)
• The MIME parts retrieved come infested with <br /> tags, since iNotes renders them in a display window (I would have used <pre>, but that's a different story. So they need cleanup

To begin I'll use the result of l_JSVars to learn about the document. Since, again, it is JavaScript, not JSON, I'll take the Rhino approach to extract a Java object.
The parts I'm interested in is in the variable named DXX which contains item that is a representation of NotesItems. True to the nature of an item, the JSON is quite elaborate, which you can see on the example of the form item:

{ "@name": "Form",
  "textlist": {
    "text": [
        {"0": "Memo"}
       ]
   }
}

For our purposes this will be flattened into a Map that has the item name as key. The Java code is quite similar to the retrieval of the Outline, the magic is in the JavaScript:

public MessageObject(String code, String rawMsg) {
    long start = System.currentTimeMillis();
    ScriptEngine engine = new ScriptEngineManager().getEngineByExtension("js");
    engine.eval(code);
    Invocable jsFunc = (Invocable) engine;
    MessageObject msg = (MessageObject) jsFunc.invokeFunction("getMessageObject", rawMsg);
    long end = System.currentTimeMillis();
    System.out.println(msg.getUNID());
    System.out.print("Time in ms:");
    System.out.println(end-start);
    return msg;
}

A little detail to stumble over is the item structure. It can be textlist/text or only text (feels like MIME Headers) and datetimelist/datetimepair/datetime or datetimelist/datetime.I haven't seen any item other than the name "0", so JSON arrays and multi-values got a little in each other's way. text as an array could have multiple values eliminating the need for an enclosing textlist, but probably I'm overlooking something, smarter people that me, who designed this, did envision.

function getMessageObject(raw) {
 importPackage(com.notessensei.demo);
 var msg;
 eval("msg = " + raw);
 var result = new MessageObject(msg.sUnid);
 var count = 0;
 var first = true;
 var curName = "";
 var curValue = "";
 for ( var id in msg.DXX) {
  var notesItem = msg.DXX[id];
  curName = notesItem["@name"]
  // Process depending on data type
  if (notesItem.text) {
   curValue = notesItem.text["0"];
  } else if (notesItem.number) {
   curValue = notesItem.number["0"];
  } else if (notesItem.datetime) {
   curValue = notesItem.datetime["0"];
  } else if (notesItem.textlist) {
   first = true;
   curValue = "";
   for ( var t in notesItem.textlist.text) {
    if (!first) {
     curValue += ";";
    }
    curValue += notesItem.textlist.text[t]["0"];
    first = false;
   }
  } else if (notesItem.numberlist) {
   first = true;
   curValue = "";
   for ( var t in notesItem.numberlist.number) {
    if (!first) {
     curValue += ";";
    }
    curValue += notesItem.numberlist.number[t]["0"];
    first = false;
   }
  } else if (notesItem.datetimelist.datetimepair) {
   curValue = notesItem.datetimelist.datetimepair[0].datetime[0]["0"];
   curValue += " - ";
   curValue += notesItem.datetimelist.datetimepair[0].datetime[1]["0"];
  } else if (notesItem.datetimelist.datetime) {
   first = true;
   curValue = "";
   for ( var t in notesItem.datetimelist.datetime) {
    if (!first) {
     curValue += ";";
    }
    curValue += notesItem.datetimelist.datetime[t]["0"];
    first = false;
   }

     } else {
      // Data type we didn't expect!
      print("Unexpected item encountered: ");
      print(notesItem["@name"]);
      print(": ");
      for (x in notesItem) {
       print(x);
       print(": ");
       println(notesItem[x]);
      }
      print("\n");
      curName = "";
      curValue = "";
     }

     if (curName != "") {
      result.addItem(curName, curValue);
     }

     count++;
    }
    print("\n Fields found: ");
    print(count);
    print("\n");

    return result;

}

When you use a Map instead of a specialized object, you need to change var result = new MessageObject(msg.sUnid); to var result = new java.util.HashMap; and result.addItem(curName, curValue); to result.put(curName, curValue);. To avoid empty fields (caveat: there's a difference between "empty field" and "field not there"), change if (curName != "") { to if (curName != "" && curValue != "") {
An interesting question to contemplate: while all values in the JSON come across as String, the structure clearly specifies the data type. Should therefore the object you create contain these data types instead of String?
As usual YMMV