Domino Development - Back to Basics - Part 5: Finding data - Collections and Search

Continuing from Part 4, this installment will clarify finding data. It, again, is different from other database concepts. Read on:
In a RDBMS your data retrieval queries are formulated in SQL, while XML, RDF or Graph databases might use sparQL, Gremlin, Cypher or JavaScript. There is no shortage of Query languages (one you should master in any case is XPath, get the full reference from the inventor himself).
On a first look a Query Language seems absent from IBM Notes. On a second look, there are two.

The IBM Notes Query Languages

Access to Notes data mostly happens using a view, so the pre-selection happens using the SELECT statement and the @Formula language. We will see how to narrow results down, just below. But you are not limited to creating a view first, a database can be queried directly using @Formula when executing Database.search("some @Formula expression"). This works like any other query language around. This flexibility comes with a price: speed. Database.search is the slowest way to retrieve data from a NSF and you shouldn't use that for your default access path (that is: the actions most users will use in a regular manner).
The second query language is the composition of a fulltext query. In its simplest form it is just some text you want to find, but it can be extended to specify the fields to look into and construct additional constraints. It takes a little to get used to it, but it is quite powerful, just one example: fish SENTENCE chips finds documents where fish & chips are mentioned in the same sentence. Important note here: make sure the FullText index is configured, so it isn't crawling slow.

Finding data

There are several ways data retrieval happens in IBM Notes:

1. Direct access
   Notes documents can be addressed using a notes:// address. The session class has a resolve(notesurl) method (not the fastest) and the database class has Database.getDocumentByUNID(id). Don't confuse it with Database.getDocumentByURL(...) which created a document based on HTML content at the specified URL. Internally the byUNID is used whenever you click on a Notes document link. It is one of the fastest access methods. There is also byID, which you should avoid, since it isn't save across replicas or copy style compaction
2. View/Folder access
   The view selection formula (for folders: action taked by users or code) defines a set of documents shown in a view. Using one of the collections (see below) you can process all those using View.getAllEntries, View.createViewNav() or by directly starting to loop through the documents using View.getFirstDocument() (or getFirstEntry()). When looping through them, any saving back of a document might remove it from the view index, so you need to fetch the next document/entry before that
3. Subsets of View/Folder
   This is the most typical query access to Notes data. To make this work the view needs to be sorted or categorized (depending on the method). A typical example in a workflow application would be a view with the selection formula SELECT Status="Pending Approval" and the first column categorized with CurrentApprover. You would retrieve documents using View.getAllDocumentsByKey(...). This method works already with sorted columns (don't need to be categorized) and can take a Vector as search term. It would then match the Vector elements to the sorted columns for search.
   However using categorization has the advantage, that you can directly address that view in an URL using ?OpenView&RestrictToCategory=search term and the view control has a matching property as well. An alternate approach is to use View.getDocumentByKey(...) to jump to the first Document/Entry and then loop until the desired end. This approach is useful when you might not process all entries or process beyond the current key (e.g. process all requests from R-Z)
4. FullText search
   Make sure you actually have a FTIndex on the database, otherwise that will be slow. The syntax takes a while to get used to. There is one FTIndex per database, even while you can limit the search to one view
5. Database.Search(...)
   The most flexible search, with the price of speed and sort order: slow and you have to sort it yourself. Don't use unless unavoidable

Depending on your needs pick one or the other

Collections

Depending on your access route choosen above, you might get one of the 4 result collections. Those are no proper Java Collections, but come in their own little API implementation, unless you use a well designed Notes API. You will find getFirst and getNext(currentOne) methods

• DocumentCollection
  The oldest and most flexible collection. Can originate from a database, a view, a search. Since you have full access to all documents in it, it is the slowest collection. It was also the first collection available in the API and is a good candidate for being replaced by a ViewEntryCollection when tuning your database. A DocumentCollection contains only data documents. Is has fancy set operations
• NoteCollection
  The NoteCollection can contain data and design Note elements and is slightly faster to build than the DocumentCollection, since it initializes a Note class only on access. So the total time "creation + access" is similar to the DocumentCollection. The main use case is access to design elements and some fancy set operations
• ViewEntryCollection
  This collection provides all (or selected) documents from a view. Instead of opening each document it gives access to the column values in the ViewEntry object, as well as the same fancy set operations you learned to love from the DocumentCollection and the NoteCollection. The ViewEntryCollection performs faster, given you have all values you need in a view column
• ViewNavigator
  Technically it isn't a collection, but the ability to point around in the view index. So you won't get any set operation to refine the result, but you get speed. You can jump from category to category (great for reading the reduce() values) and run through an entire view very very quickly. The ViewNavigator is a main reason to plan your views well

Next up: Better save than sorry - Security