From Blogsphere to a Static Site (Part 4) - Comment backend

The blog needed a comment function. While there are social options around (Facebook, Disqus etc), I decided I want to roll my own. Partly because I want tighter control and partly, well, because I could. My comment backend would:

• Provide a REST API to create comments in a JSON structure. The comment body will be Markdown. Reading would provide comments in ready to use HTML (I hear howling from the API crowd). No delete or update functionality
• Cleanup content considered harmful (code injection) and optional sport Captcha
• Store all content in a NoSQL database, in my case CouchDB (or Cloudant with its 20G free plan)
• Cache all queries for comment in an online cache to limit calls to the database
• Initially run on Domino, later on liberty or the raw JVM
• Initially also update Domino using a web service - so during transition no comments would get lost

In its initial incarnation the Comment servlet is a OSGi plugin that listens to the /comments URL implemented as Wink servlet. So the class of interest is the one defining the service. We have one method for post, one for get and a helper function

/**
 * Wink implementation of Comment service
 */
@Workspace(workspaceTitle = "Blog Comments", collectionTitle = "Create or display comments")
@Path(value = "/comments")
@Produces(MediaType.TEXT_HTML)
@Consumes(MediaType.APPLICATION_JSON)
public class CommentService extends CommentResponse {

    private final Logger logger = Logger.getLogger(this.getClass().getName());

    @POST
    public Response createComment(@Context HttpServletRequest request) {
        final Monitor mon = MonitorFactory.start("CommentService#createComment");
        String result = "Sorry I can't process your comment at this time";
        ResponseBuilder builder = Response.ok();

        try {
            InputStream in = request.getInputStream();
            BlogComment comment = BlogComment.load(in);
            in.close();

            if (comment != null) {
                this.captureSubmissionDetails(request, comment);
                result = CommentManager.INSTANCE.saveComment(comment, true);
            } else {
                builder.status(Status.NOT_ACCEPTABLE);
            }
            builder.entity(result).type(MediaType.TEXT_HTML_TYPE);

        } catch (Exception e) {
            String errorMessage = e.getMessage();
            builder.entity((((errorMessage == null) || errorMessage.equals("")) ? "Undefined error" : errorMessage)).type(
                    MediaType.TEXT_HTML_TYPE);
            Utils.logError(this.logger, e);
        }

        mon.stop();
        return builder.build();
    }

    @GET
    public Response getComments(@QueryParam("parentid") final String parentid) {
        Response response = null;
        final Monitor mon = MonitorFactory.start("CommentService#getComments");
        final ResponseBuilder builder = Response.ok();
        final Collection<BlogComment> bc = CommentManager.INSTANCE.loadComments(parentid);

        if ((bc == null) || bc.isEmpty()) {
            builder.status(Status.NO_CONTENT);
        } else {
            response = this.renderOutput(bc, "comment.mustache");
        }
        mon.stop();
        return (response == null) ? builder.build() : response;
    }

    private void captureSubmissionDetails(HttpServletRequest request, BlogComment comment) {
        final Monitor mon = MonitorFactory.start("CommentService#captureSubmissionDetails");
        try {
            @SuppressWarnings("rawtypes")
            Enumeration hn = request.getHeaderNames();
            if (hn != null) {
                while (hn.hasMoreElements()) {
                    String key = hn.nextElement().toString();
                    comment.addParameter(key, request.getHeader(key));
                }
            }
            @SuppressWarnings("rawtypes")
            Enumeration pn = request.getParameterNames();
            if (pn != null) {
                while (pn.hasMoreElements()) {
                    String key = pn.nextElement().toString();
                    String[] values = request.getParameterValues(key);
                    comment.addParameters(key, values);

                    if (key.equals("referer")) {
                        comment.setReferer(values[0]);
                    } else if (key.equals("user-agent")) {
                        comment.setUserAgent(values[0]);
                    }

                }
            }
            @SuppressWarnings("rawtypes")
            Enumeration an = request.getAttributeNames();
            if (an != null) {
                while (an.hasMoreElements()) {
                    try {
                        String key = an.nextElement().toString();
                        comment.addAttribute(key, String.valueOf(request.getAttribute(key)));
                    } catch (Exception e) {
                        // No action here
                    }
                }
            }
            comment.addParameter("REMOTE_HOST", request.getRemoteHost());
            comment.addParameter("REMOTE_ADDR", request.getRemoteAddr());
            comment.addParameter("REMOTE_USER", request.getRemoteUser());

            // Needed for Captcha
            comment.setRemoteAddress(request.getRemoteAddr());
        } catch (Exception e) {
            Utils.logError(this.logger, e);
            // But no further action here!
        }
        mon.stop();
    }
}

The two interesting lines in the class above are CommentManager.INSTANCE.saveComment(comment, true); and CommentManager.INSTANCE.loadComments(parentid);, with the former saving a new comment and the later loading the list of comments. Both use the CommentsManager Singleton to access comments. The key component is a Google Guava cache and the Ektorp CouchDB library

Setting up the cache

 class CommentLoader extends CacheLoader<String, Collection<BlogComment>> {

        private final Logger logger = Logger.getLogger(this.getClass().getName());

        @Override
        public Collection<BlogComment> load(String parentId) throws CommentException {
            Monitor mon = MonitorFactory.start("CommentManager#CommentLoader#load");
            CouchDbConnector db;
            List<BlogComment> result = null;

            try {
                db = CommentManager.this.getDB();
                ViewQuery query = new ViewQuery().designDocId("_design/comments").viewName("byParentId").key(parentId);
                result = db.queryView(query, BlogComment.class);
            } catch (MalformedURLException e) {
                Utils.logError(this.logger, e);
                throw new CommentException("Comment retrieval failed", e);
            }

            if (result != null) {
                Collections.sort(result);
            } else {
                throw new CommentException();
            }
            mon.stop();
            return result;
        }
    }

        // Get comments from the database
        CommentLoader loader = new CommentLoader();
        this.commentCache = CacheBuilder.newBuilder().maximumSize(1000).recordStats().expireAfterAccess(8, TimeUnit.HOURS)
                .build(loader);

Save a comment

public String saveComment(BlogComment bc, boolean validate) {
        String key = this.config.CHECKCAPTCHA ? this.config.RECAPTCHAKEY : null;
        boolean isValid = bc.isValid(key);
        String result = bc.getValidationResult();

        if (isValid || !validate) {
            CouchDbConnector db;
            try {
                db = this.getDB();
                db.create(bc.getId(), bc);
                // Now save into the cache
                Collection<BlogComment> cachedComments = null;
                cachedComments = this.commentCache.getIfPresent(bc.getParentId());
                if (cachedComments != null) {
                    cachedComments.add(bc);
                }
            } catch (Exception e) {
                Utils.logError(this.logger, e);
                result = e.getMessage();
            }

            // Now notify the legacyBlog
            BackgroundWebServiceCall b = new BackgroundWebServiceCall(bc);
            this.executor.execute(b);
        }
        return result;
    }

Load comments

public Collection<BlogComment> loadComments(final String parentId) {
        Collection<BlogComment> result = null;
        if (parentId != null) {
            try {
                result = this.commentCache.get(parentId);
            } catch (ExecutionException e) {
                Utils.logError(this.logger, e);
            }
        }
        return result;
    }

Getting a gravatar image

public String getGravatarURL() {

        if ((this.gravatarURL == null) || this.gravatarURL.trim().equals("")) {
            if (this.eMail != null) {
                String emailHash = DigestUtils.md5Hex(this.eMail.toLowerCase().trim());
                this.setGravatarURL(GRAVATAR_URL + emailHash + ".jpg?s=" + GRAVATAR_SIZE);
            }
        }

        return this.gravatarURL;
    }

Create HTML from Markdown

private void createHtmlBody(String markdownBody) {
        PegDownProcessor p = new PegDownProcessor();
        this.htmlBody = p.markdownToHtml(HTMLFilter.filter(markdownBody));
    }

Check for valid Captcha

public static boolean isValidCaptcha(String captchaKey, String remoteAddress, String challenge, String response) {
		boolean result = true;
		// We only test if we have a remote address and the captcha switch is on
		if (remoteAddress != null && captchaKey != null) {
			ReCaptchaImpl reCaptcha = new ReCaptchaImpl();
			reCaptcha.setPrivateKey(captchaKey);
			ReCaptchaResponse reCaptchaResponse = reCaptcha.checkAnswer(remoteAddress, challenge, response);
			result = reCaptchaResponse.isValid();
		}

		return result;
	}

For the full details of the implementation, including the background task talking to Domino, you need to wait for the source release on Github. Next stop: the comment front end