Usability - Productivity - Business - The web - Singapore & Twins

A filtering proxy server with vert.x


You have this nice application running in your (cloud or on premises) environment and then a big scare hits. Suddenly you need to remove or mask different streams of data depending on all sorts of conditions your legal department is torturing advising you.

Until your applications natively can do that, you might resort to a content filter that sits as a proxy between you and the application (technically it is a reverse proxy, but that's fine print).

To explore the feasibility of such an approach I created SampleProxy based on work of Julien Viet using vert.x as my runtime environment.


  • Needs to be a content filter, not a URL blocker
  • Need to provide functionality for practical use out-of-the-box, but needs to be extensible (configuration over code)
  • Need to be able to filter HTML, JSON, XML and Text. No need to filter binary formats. Contemplating about JavaScript (you could use the text filter for that)
  • Filter based on mime-type and URL as standard, but extensible to use anything in the request or reply to decide what to filter
  • Configurable FilterChain: a filter decides what to filter (with the mime-type as minimum condition) and hands actual filter operation to a chain of subfilters that do the actual stream manipulation
  • configurable subfilters. E.g. a filter that can remove JSON nodes from JSON data should read the qualifier from a configuration, so the same filter class can be reused for different filter purposes
  • CSS isn't on the radar yet, but contributions would be happily accepted

The flow

Flow from browser to proxy to application and back

Things I learned along the way

There are always a few lessons to be had, here are some from this project:

  • http is a chunked beast. When you send larger amount of content, probability approaches 1 that your server uses chunked - until HTTP/2 resolves us from it. A hard choice needs to be made to either use a stream based processing of a chunk (think SAX) or collecting the Junks to be able to process a DOM. To be fully flexible I opted for a DOM/Object based approach, but you are free to create whatever you deem necessary
  • Jsoup is a reliable HTML parser. It supports CSS selectors that make addressing HTML elements a breeze. Solves one of the hardest problems: targeting
  • Targeting JSON data is much harder that it needs to be, the very moment Arrays appear in your JSON structure. There is RFC6901 JSON Pointer, but it targets exactly one element, while a typical use case would be: ?from the list (array) of discussion posts, pick the list of comments and those who have an eMail, mask them?. So I implemented 2 variations: a simple path style address /discussion/posts/comments/email which automatically traverses arrays and an XPath based approach where I convert JSON to a strict XML syntax and back. More detail here, examples in a future post

Items on the ToDo list

  • Better documentation
  • Code cleanup
  • Tests
  • Deploy to Heroku button
  • More filters

Your turn

Go check it out and let me know what you think! (Yeah - documentation needs some work).

Caveat (a.k.a disclaimer): this is a prototype and work in progress, YMMV!

Posted by on 2018-03-04 11:18 | Comments (0) | categories: Java Heroku vert.x

Query a JSON object like XPath in Java

I'm fond of XML especially XSLT and the related XPath query possibilities.
Moving into the wonderworld of JSON I'm missing those platform independent query and rendering options.
For templating Mustache comes close, especially in regards of platform support, but XPath still has to find its match in the JSON world.

The closest equivalent is JSONPath (also available for Java ) with a nice interactive playground.

I was looking at adopting it for one of my projects written in vert.x.
JSON isn't native to Java and there are several competing implementations around JSON: Google's GSON, Jackson
and vert.x core JsonObject (there are probably more).

So I was wondering: What if I could use XPath and by extension XSLT directly on JSON? The first step there would be a save way to transform JSON to XML and back.
Using vert.x JsonObject and JsonArray as starting point,
I had the following considerations:

  • I can't use XML tags to map to JSON labels, since labels can contain any character, XML tags are restricted
  • The resulting XML only needs 2 elements: Element and Array
  • An element can have a scalar value, my first test deals with strings only, or can contain another element or array
  • An array does not have a value. It can contain zero or more scalar values and/or zero or more arrays or elements

So I build a small prototype. A simple class that reads a json file from the command line, prints it to the console, transforms it to XML, prints it to the console, transforms it back and prints it again.
The prototype depends on vert.x core.

The result can be found in this gist or read more.

Read more

Posted by on 2018-02-28 11:50 | Comments (0) | categories: vert.x Java

Spring Boot and Salesforce Canvas

The challenge

Salesforce canvas offers a capable integration point between Salesforce and external applications to surface them inside the Salesforce UI. One of the aspects is establishing identity. There are two options: OAuth and a signed request. I?m looking at the later. A signed request posts (as in HTTP POST) a digitally signed JSON request to the external application.

When all you need is a single page, validating the request and returning the result is all that needs to be done. It becomes trickier when you want to navigate in the application and when that application runs in the cloud with multiple load balanced instances, so you might end up on a different instance mid-flight.

Most of the sample code available uses the sophisticated Canvas JavaScript SDK, so I decided to focus on backend code here.

The project

All the code can be found on Github including sources. Feel free to open issues. To try it out you can deploy it to Heroku (button in the Readme on Github) and configure a Canvas app in Salesforce. Use https://yourappname.herokuapp.com/sfdcauth/hw as Canvas URL

The approach

Build a Spring Boot application that provides an authentication endpoint suitable for a Canvas POST and other endpoints that only allow authenticated access. The security will be provided by Json Web Tokens a.k.a JWT or RFC 7519.

As added challenge: The application will require standard link and form based navigation, so we can't rely on AJAX to provide additional ?stuff? into the requests from/to the server. And yes - needs to be able to run on Heroku on multiple instances (Dynos in Heroku language) without the sticky session feature switched on.

Lessons learned

  • Spring Boot performs magic. Sometimes a little too much magic
  • Base64 isn't Base64. The Java8 decoder works different from the Apache Commons, so the signed checksums don't tally. Pain in the ?proverbial? to get the validation code from the SDK running
  • Spring security is well thought through. I like the approach: the very moment you add it to a project, all is closed until you specifically open it
  • You can't stuff the entire JSON coming from Salesforce back into a cookie
  • JSON in Java is still a pain (Bearable with JsonNode)
  • The information around custom error pages in Spring is confusing. It seems to have evolved over time and conflicting information is presented
  • Json Web Tokens (JWT) rock
  • Cookies behave different in Salesforce canvas than in regular iFrames. In an iFrame http cookies are considered ?native? to their environment, in canvas they seem to run as third-party cookies (unless I miss something). Stole a few hours tracking the issue down
  • Avonni Creator is a nice tool to play with Lightning Design System layouts

As usual YMMV

Posted by on 2018-02-20 05:34 | Comments (0) | categories: Salesforce Heroku Java

Running NodeRED on Heroku with Salesforce

I am a big fan of NodeRED, so I was delighted to see nodes available to connect to Salesforce.
Since the package was a little behind and support for platform events was still missing, I had a chat with Jeff and we agreed that I would maintain the package moving forward.

I updated the package dependencies to current version and made two main changes to the functionality:

  • allow credentials to be read from environment variables. This allows for easy deployment to Heroku
  • Support for Salesforce platform events

Deployment to Heroku is a breeze, but requires a few steps that I document here.
The instructions require a basic understanding how Heroku works, so you might want to check out a tutorial first. There are some challenges to overcome:

  • I want to be able to test local and on the server
  • Storage on Heroku is ephemeral. Any changes written to the server's file system get reset on restart
  • Deployment should happen via version control


  • Current version of NodeJS installed (I usually stick to the LTS version)
  • GIT client installed (or GIT as part of your favorite IDE)
  • Heroku CLI (a.k.a Toolbelt) installed. Install using npm install -g heroku-cli

Project setup (Command line work)

  • Create a new directory NodeRedDemo and change into it
  • Initialize a new project npm init - fill in suitable defaults
  • Create a file .gitignore (content see below)
  • add NodeRED and the Salesfore nodes npm install -s node-red node-red-contrib-salesforce
  • add a line into package.json (see sample below) in scripts: "start": "node-red --settings ./.node-red/settings.js --userDir ./.node-red"
  • create the directories scripts and .node-red
  • copy the settings.js file from node_modules/node-red to .node-red - We will edit that file later
  • initialize git: git init
  • add initial files to git: git add --all and git commit -m "initial creation"
  • login to heroku heroku login
  • add Heroku: heroku create
  • deploy: git push heroku master

Your application should be running on Heroku now. To learn how to use NodeRED, check out a tutorial. Besides running on Heroku you can start the application on local too. Just use npm start and load localhost:1880.

What is missing: protect your NodeRED editor, connect to Salesforce and of course: create your flows.

Keep in mind: every time you redeploy or restart the app, anything stored, including your flow definitions, gets lost.
For permanent results I create my flows in the local instance of NodeRED and commit the flow file (in the .node-red directory) to git, so it becomes part of the deployment.

Read more

Posted by on 2018-02-14 11:42 | Comments (0) | categories: Salesforce Heroku NodeRED

Excel in Continuous Integration

Business Users like Excel. Besides its original use case of calculating, lists are a favourite use case. They also serve as poor man's requirement and bug tracker, so they siep into software development too.

While Excel sheets are great for interaction, they are a beast for anything automation. The irony of it: Modern Excel files (the xslx flavour) are nothing less than zip files with XML content. However that format is, let's say, [insert expletives here].

From an XML representation I would expect something like <cell row="23" col="44">Some value</cell>. However that's not what Excel does. Rename an xlsx to zip and see for yourself. Also (which makes sense for Excel itself) empty cells are not represented in XML.

Cutting a long story short, an Excel file or its XML representation poses some challenges:

  • XML format is not very suitable for automation, like generating reports using XSLT
  • Excel automation only runs on Windows (and when you run headless, you trade the head for a headache). That makes it a no-go for most automation server environments
  • Empty cells are absent from the XML (a variation of "not suitable")
  • Cross reports with other files (e.g. logs in XML format) is hard

To overcome these limitations I wrote Excel2XML. It is a little Java command line utility that converts Excel into a more digestible XML format. I used Microsoft's contribution to the Apache POI project to read the file. It has the following functions:

  • Extract workbooks in one or separate files per worksheet
  • Ignore all formatting
  • Computed cells return their last result values, unless it is a formula error, then the formula is returned
  • The first line of each sheet is treated as column headers, which are extracted as columns/column elements
  • Each cell has a column, a row and a title attribute. The title reflects the value from the first row. This allows in XSLT to query the title instead of relying on the column number. Reordering, adding or removing columns won't kill your XSLT stylesheet that way
  • Optional empty cells can be generated with an attribute of empty="true"
  • Runs on Java8 completely from command line
  • Calling it without parameters outputs the exact syntax of options

The full syntax: java -jar excel2xml.jar -i somefile.xslt [-o somefile.xml [ ]-e] [-s] [-w3,4]

  • -i the input file in xslx format
  • -o the output file. If missing same name as input, but extension xml
  • -e generate empty cells. If missing: cells without data are skipped
  • -s generate a single file for the whole workbook. If missing: creates one file per sheet
  • -w comma separated list of sheets to export. Starts at 0. If missing: exports all sheets

Head over to the git repository and grab a release. Let me know what use you found. As usual YMMV

Posted by on 2017-10-02 10:28 | Comments (1) | categories: Continuous integration XML

Static APEX code analysis with PMD

Coding standard are only as good as they are followed. One venerable tool to check source code is PMD. When you, like I did, wonder what acronym PMD represents, you will be disappointed. PMD supports the usual suspects like C, C++, C#, Fortran, Java, JavaScript, Go, Groovy, XML... as well as Apex and Visualforce.

The default ruleset for Apex contains 34 rules and are the ones also used by CodeClimate

PMD is a Java application and you have a number options to run it:

  • As a command line application
  • Using ANT or Maven
  • In your CI platform (using command line, ANT or Maven )
  • Linking your Github (or Gitlab, Bitbucket) repository to a service like CodeClimate

PMD supports a series of output formats like plain, csv, html (different flavours) and XML. When you understand XSLT, the XML format allows you to render whatever report you deem fit

Read more

Posted by on 2017-07-26 12:55 | Comments (1) | categories: Salesforce Apex XML

From Blogsphere to a Static Site (Part 5) - Comment front-end

In Part 4 I described the comment backend. This installment sheds a light on the comment front-end.

Comments can be tricky. One lesson I learned early: When your comment form is standard HTML form, it attracts spam like a light bulb attracts moths. So the requirement were:

  • The original blog entry should not contain any HTML form. It should be loaded on a button click using JavaScript. Nota bene: this isn't hide/show, but actual manipulation of the DOM
  • The dynamic form shall not contain a POST URL, but submission should be in JavaScript - keeps a lot of the scumbags out already
  • Submission should be secured with a Captcha
  • Some formatting should be allowed. I opted for a Markdown editor with preview capabilities

The first component is the placeholder for the existing comments and the button showing the comment form:

<a name="comments"></a>
<div class="well well-raised" style="text-align : center">
  <button class="btn btn-lg btn-info" data-toggle="collapse" data-target="#commentform_{{UNID}}" type="button">
    Add your comment...  <span class="glyphicon glyphicon-comment"></span>
<div id="commentform_{{UNID}}" class="collapse"></div>
<div class="well well-raised">
  <ol id="commentList">
      {{#gravatarURL}}<img src="{{.}}" class="gravatarimg" /> {{/gravatarURL}} posted by <b>{{author}}</b> on <i>{{createdString}}</i>:
      <br /> {{& comment}}
      <hr style="clear : both" />
    </li> {{/comments}} {{^comments}}
    <li id="nocomments">
      <h5>No comments yet, be the first to comment</h5>

The second component is the comment form, implemented as mustache template - one of the reasons I picked Mustache: runs on the server and the client in tons of languages

<form title="Comment form for blog discussion" onSubmit="return addComment(this,'{{recaptchaid}}','{{parentId}}')" class="form-vertical well well-raised">
    <legend>Add your comment</legend>
    <p>Please note: <b>Comments without a valid and working eMail address will be removed.</b>
      <br /> This is my site, so I decide what stays here and what goes.</p>
     <div class="control-group" id="commentcontrol">
      <label class="control-label" for="Commentor">Name (required, published)</label>
      <div class="controls">
        <input class="input-xlarge focused" id="Commentor" size="30" accesskey="n" name="nameAuthor" />
      <label class="control-label" for="Email">eMail (required, not published)</label>
      <div class="controls">
        <input type="eMail" class="input-xlarge focused" id="Email" size="30" accesskey="n" name="txtEmail" placeholder="A working eMail please!" />
      <label class="control-label" for="webSite">URL (optional)</label>
      <div class="controls">
        <input type="url" class="input-xlarge" id="webSite" size="30" accesskey="n" name="txtWebSite" />
      <div class="controls">
        <div id="wmd-panel" class="wmd-panel">
          <table style="width : 100%" border="0">
              <td style="width : 50%; vertical-align : top">
                <label class="control-label" for="wmd-input">Your Comment (Use markdown like <a href="//stackoverflow.com/editing-help" target="_blank">Stackoverflow</a>)</label>
                <div id="wmd-button-bar"></div>
                <textarea class="wmd-input" id="wmd-input" name="Body"></textarea>
              <td style="width : 50%; vertical-align : top">
                <label class="control-label">Preview</label>
                <div id="wmd-preview" class="wmd-panel wmd-preview"></div>
      <div class="controls" id="captchadiv">Captcha here</div>
      <div class="form-actions">
        <button id="commentsubmit" type="submit" class="btn btn-primary btn-large">Post your comment</button>
    <div class="alert alert-block" id="alertContainer" style="display : none">One moment please, submitting comment...</div>

The whole mechanism gets to work with just a few jQuery JavaScript functions (Vanilla JS would work too, but I had JQuery already for the social buttons, so I reused that) and the respective JS files:

Read more

Posted by on 2017-07-20 10:58 | Comments (0) | categories: Blog

Ownership flow of customer community created cases

In a previous entry I mapped out sharing abilities the Salesforce platform is capable of.

In this entry I will put these capabilities to practial use. The scenario starts with a customer logging into the customer community and raising a ticket. Client representative, the partner and support staff need/want access to the case. The case needs to flow along the same processing as cases created by other means (call, email, IoT). The following diagram explains the flow (click for a larger version):

Customer Portal Created Cases Ownership flow

The diagram has been rendered using Websequencediagrams based on a plain text file:

Read more

Posted by on 2017-07-16 09:53 | Comments (0) | categories: Salesforce

Using FontAwesome in your VisualForce pages

I'm a big fan of Font Awesome (including backing Version 5). It allows to add icons without the overhead of loading tons of small graphic files. In a recent internal project I had to integrate FontAwesome into a Salesforce VisualForce page

Installing FontAwesome

In theory you could load the font from a CDN. However based on security settings this might or might not work. So I decided to add FontAwesome as resource to my project:

  1. Download FontAwesome 4. Or support the team and use Version 5
  2. Add the zip file as static resource to your setup:
    Static Resource for FontAwesome
  3. Make sure you have set the cache control to public, so the font doesn't need download after first use.

Adding to the VisualForce Page

The font gets added using a stylesheet that is included in the zip file added as a resource in the previous step. the apex:stylesheet element takes care of that. The only catch: the zip file is internally "expanded" OS/X style with the file name (sans extension) as directory name. So the tag needs to look like this:

  <apex:stylesheet value="{!URLFOR($Resource.fontawesome,'/font-awesome-4.7.0/css/font-awesome.css')}"/>

Using Icons

This follows the standard approach: you create an empty &lt;i&gt; element and use the class property to specify icon and options. Like this:

  <i style="color: red" class="fa fa-user-circle fa-2x" aria-hidden="true"></i>

Make sure you have a look at the full selection of icons available.

As usual: YMMV

Posted by on 2017-07-14 08:43 | Comments (1) | categories: Salesforce

Salesforce Community License Overview

To facilitate collaboration across corporate boundaries, Salesforce offers Salesforce Communities. The licence for communities come in three flavours: customer community, customer community plus and partner community. There are similarities and differences between the entitlements.

The biggest difference can be seen between customer community and customer community plus. There's a simple rationale behind it: a customer community account uses simplified sharing access, so the computational load to determine access of potentially millions of users stays reasonable and easy to grasp for administrators:

Community License Overview

All licences, full Salesforce or one of the community licence types, access the same data, in the confines of the entitlements. So it's NOT different products, just different permissions on the Salesforce platform.

Posted by on 2017-07-10 09:40 | Comments (0) | categories: Salesforce