Continuing from Part 3, this part is typically the hardest to understand when coming from an RDBMS background. So take your time.

Domino Views are different

In Domino data is typically accessed via a view, but views are different than the ones you know in an RDBMS. The following table should provide an rough overview.

Item	RDBMS	Domino
Data	Defined in a database schema. Data is contained in tables. All records in a table are uniformly the same. Fields without values are there but empty. Different types of data require different tables. New data requires ALTER TABLE statements wich affects existing data (and often requires down time). Each column can have one value	Data is contained in documents. Documents are schema free (there is a Meta Schema). Items in documents (the closest to a column in a table) can have multiple values. Each document can have a different set of items. New data is added as needed, no downtime or change of existing documents required
Views	Selection of rows, often accompanied with a JOIN operation to denormalize normalized data. Can be all data of a table or a subset, picked by a WHERE part in the SELECT statement. By definition views don't contain any data, but pull them ad hoc from the participating tables. Various RDBMS systems use indexes on keys used in SQL statements to improve performance. The ad-hoc nature of the queries offers maximum flexibility at CPU and I/O cost. There is a whole industry around SQL Query Optimization (and there is the whole story about SQL Injection attacks even by mothers)	are pre-created and updated by a view indexer task when documents are created or updated. They do occupy space inside an NSF (there are discussions in development to externalize them to reduce backup load and allow more I/O tuning). Since they don't require a document to be opened, they are fast (unless you use performance killers - see below) Each column in a view shows data. Using Domino designer that data can be different for each row (defined by @Formula Since items can have multiple values, they can appear more than once in a view (controlled by the view design) Different document types (e.g. a customer entry and an order entry) can show up in the same view without the need for a JOIN operation (which anyway isn't a feature of a Notes view - this is one more reason why Notes favours inheritance over relation) Accessing a view using a ViewNavigator is very fast A view can contain data that is computed and not contained in the documents. Typical uses are translations of status codes to human readable text)
Hierarchy	SQL Tables and views are flat by definition. Some SQL extensions (e.g. Oracle) allow hierarchical queries, but the result will be a flat query result.	views can be flat or hierarchical A common use for hierachical views is the option to categorize a column value. This operation will sort the column and show each value once only above the result rows (in the UI typically with a twistie icon) A categorized column also can perform some additional simple Reduce operations (albeit not distributed) Programmatic categorized views are typically used with @DBLookup; @DBColumn; getAll[Documents|Entries]byKey Multiple columns can be categorized Typically the categorized columns are the first ones on the left, but that isn't necessary Subcategories can be created by having the item value containing a backslash. If an item contains more than one value, the document will appear in multiple categories. An alternative approach turns multiple values into subcategories: @Implode(fieldname;"\\") Besides categorization there is a secondary hierarchy mechanism available: the response hierarchy. Defined by a view property, response documents (of all levels) are shown automatically below their respective main document. A special column can be defined that is shown only for response type documents (containing a valid $REF item). In this case only the columns left of this are shared with main documents, columns right of it are ignored for responses and only the response column is shown (check the discussion template for how it looks)
Sorting	Sorting happens using an ORDER BY clause. For resorting a new select clause needs to be issued. RDBMS systems allow indexes to be created to improve sorting speed.	Sorting is a property of a view column. It will increase the view index size and thus I/O and reindexing time Sorting happens from left to right Sorted columns don't need to be visible in the view (there is a property to hide columns conditionally or in any case) Columns can be resorted in the UI when the respective property is set. The custom sorting sequence is also available as URL parameter A common design mistake (creating index data unused): make every column sorted, even if the sorting has no more effect on the result sequence and/or make every column resortable even if users might not use it. For the later case there is a remedy: set a column property to defer the creation of the alternate index until it is requested the first time
Definition	Views are defined in SQL using any editor of your choosing and then uploaded/executed in the database. Database admin rights are required	Views are defined in Domino Designer or even the standard Notes client. Users can be allowed to create "private" views only visible to themselves, or at access level Editor views that can be shared. Columns are defined by picking item names from a list, predefined functions or writing @Formula a LISP like language (easy to learn, quite powerful to use). Brave souls have been spotted using DXL to write view definitions in XML and upload them into designer (but you don't want to do that)
Selection	View entries are selected by WHERE in SQL	The selection statement also start with SELECT, but uses the @Formula language for picking the documents All documents (typically in a view named ($All)) are selected by SELECT @All More samples: all customer documents by SELECT Form="Customer", incomplete workflows by SELECT @IsMember(Status;"Submitted":"In Progress") Overly complex select statements slow down view computation and you might need to employ some brute force view tuning A performance tip is to compute the criterion for selection inside the document to result in a field value @True|@False and then limit the selection formula to SELECT showInView_Pending
Performance killers	There is nothing better to make your hardware seller more happy than code that violates performance considerations. In SQL the typical performance killer are OUTER JOINS especially over many large tables. Missing indexes are another cause	Since Notes views contain actual data, they need to be treated a little different. There are a few killers around: Performance killer number one is the use of time based view selection formula: @Now; @Today; @Yesterday; @Tomorrow. In one word DON'T. IBM isn't so strict, but I am. You do have options The second killer is the use of @IsResponseDoc as part of a view selection when you end up with only a few parent documents, show documents in response hierarchy and many "hidden" responses since the parent isn't displayed. Better use @AllDecendants, check the explanation for more details As mentioned above: be stingy with sorting of columns, both presorted and optional. Make sure if you define optional sorting, that you defer creation until first use Overly complex view selection formulas and too many views slow down the view indexing process, so take only as much as you really need. In classic Notes we often find countless variations of views for display. In XPages you can often combine them (given they differ only in columns) and use the XPage for the variation in look and feel Last not least: your code. Let the view indexer do its job. If you use view.refresh() everywhere "as a precaution" you pay dearly with sucking performance. A view gets locked when reindexed, so 100 users all executing a view.refresh() lock the view 100 times sequentially

Next up: Finding data - Collections and Search

Continuing from Part 2 this installment will introduce you to the four types of documents you can find inside an NSF:

1. Documents
2. Responses
3. Response to Responses
4. Profiles

Profile Documents

All document types follow the same rules as outlined in Part 2, but differ in the way they are created or accessed. Short of profiles all documents typically get accessed in one or more views, by users or your code. Profiles on the other hand, never show up in a view, they are only accessed using some code. The easiest case is @Commmand([EditProfile]"NameOfAForm") for a System profile and @Commmand([EditProfile]"NameOfAForm";@UserName) for a user profile. Notes keeps an internal index for profiles and they are typically cached in memory, so accessing them is superfast (and quite a headache if you fall into the trap to use them for frequently updating information). Their intended use is storing configuration and selection options, so @GetProfileField(FormName;FieldName) has rapid access to options. In XPages applications the need for profiles is diminished, since you typically would store such configuration options in the Session and Application Scope where you load them once.

Document hierarchies

The different "Document types" are actually "only" form properties (One can see how forms and documents are points of confusion). Inside a document the presence of the $REF field (containing the UNID of the parent) makes a document a response. If the parent document happens to be a response itself, the document then is considered a "Response to Response" All Notes APIs have methods to create or query a Parent - Child hierarchy. A good overview can be found in this Ytria TechLab. To my knowledge a Response-to-response document doesn't contain a direct reference to the root document
The form properties are used when you create a document in the Notes client to automatically establish Parent-Response relationships. A typical mechanism used in response document forms is to enable inheritance (that feature can also be used for standard documents, it isn't limited to responses!): When the document get created in the context of the parent (think: you read an entry in a discussion and press reply) all items from the parent document, that have fields in the response form with the same name as the item get copied into the response document (good fun if item and field don't have the same data type). Additionally all formulas computation in the response get executed against the parent, but stored in the response. This happens once at creation, not thereafter. To keep fields in sync you would use @GetDocField($REF;"SomeFieldName") to obtain the latest value (there are methods in LotusScript, JavaScript and Java too).
Sounds complicated? Go and create a discussion database (the template is on your Notes client) and play with it for a while. All this leads to an important Notes concept:

Inheritance over Normalization

Database normalization is a corner stone concept of RDBMS. If RDBMS is all you ever used and you are stuck in the idea that databases = RDBMS ( really?), this will be hard to swallow.
Normalization is an abstraction that is suitable and needed for an RDBMS, it isn't a real life fact or an absolute necessity. If you blindly apply Normalization to XML Database, document databases, Graph databases you create more problems than solutions. So free your mind and follow:
Let's look at a typical normalization example: an invoice: You have one customer table, one invoice table, one items table and one products table. In your invoice table you point to the customer table, in the item table to your invoice table and your product table. Life seems good.

But then pesky reality kicks in: Customers move, but legal demands that existing invoices don't change (commonly changing a legal document is called forgery). Product prices change, or sales negotiated a different price. Customers might call your articles different from your product table etc. So you adjust your diagram:

The attributes in red are inherited from the respective tables. This inheritance in an RDBMS is entirely done in code, in (classic) Notes it could be done through a form property. So inheritance is also quite common in RDBMS based applications too.
In Notes it is rather the rule than the exception. Retrofitting an RDBMS normalization onto a Notes application might not bring the benefits you might expect, so keep in mind:
Notes favours inheritance over normalization.
Interestingly when you want to export data, e.g. for data interchange you hardly generate one table/file per RDBMS table, but you de-normalize the data for transport. In Notes this tandem of normalization/de-normalization isn't necessary since you deal with a (self contained) document already.

Items have multi-values and can be grouped

A typical way in a Notes application to handle parent-child records is the use of a group of related multi-value fields (in the form), resulting in a set of items in the document with the same number of values. You might have one address field (which can have many lines - much more flexible than a text column in an RDBMS) and a set of multi-value fields like: ProductID, description, Price, Quantity and SumPrice. Since the (classic) UI didn't make it very easy to keep those in sync, a module called Table Walker had been made available on the Sandbox. I later created Tablewalker for XPages as part of the XPages tutorial (with enlightened bug fixes courtesy Matt White). Keeping the line items inside a document makes transporting, processing or securing them much easier, however it makes querying the line items a bit more difficult (more on that when looking at views). You can take a similar (actually more powerful from a modelling point of view) approach when using DB/2 PureXML. DB/2 PureXML and XPages are actually a match made in heaven (or was that the other place starting with h?), which is the subject to another story, another time.
Remember (before you start bitching about parent-child relations): the explanations above are guidelines, there are valid cases where RDBMS like separation of properties and detail rows into separate documents is a better approach. You know how to normalize, so I don't need to explain that part

A Document can save directly to an RDBMS

There was an ill-fated attempt to use DB/2 as storage destination for NSF data called NSFDB2 which unfortunately predated PureXML capabilities (otherwise the story might have been different), and it shall not be spoken about it anymore (support for it has ceased anyway). The tools here, available since R5 are DECS (Domino Enterprise Connection Service, included on every Domino server and LEI ( Lotus IBM Enterprise Integrator, an IBM product you need to buy). Filling in some simple forms in the configuration database can link your Notes forms (and partially views) to backend data from various databases or ERP systems. Once you got a grasp of these tools, some creative usage patterns become possible ( more details here about Domino and RDBMS). Of course you can use code (ODBC, JDBC or LCLSX) your data transfer or use the IBM Swiss Data Knife for moving data around.

Next up: Domino views are different

Continuing from Part 1 you now know that everything in Notes is stored in a Note. To further understand how Notes "ticks" some light needs to be shed on the relation between forms and documents.
In the simplest way to look at it, a form acts like the schema to be used to populate a document with values. The fields one fills into a form get stored as items into a document. This explanation is good enough for starters, but warrants a closer look to fully appreciate Notes' flexibility. In a RDBMS a record is "unbreakable" linked to the containing table by the database schema definition. Each record has each column defined in the schema. Notes ticks differently. The link between a form and a document is established by a Notes item with the reserved name "Form". When you open a form, fill it in and save it, automatically a item with the name of the form gets created. When that document gets opened, it will pull that form for display.
That link is not static: it can be overwritten in many ways:

• In the Notes client using View - Switch Form ...
• Though a field with the name form containing a formula
• Code in an event
• Code in an agent
• A form formula in a view
• A Smart button

A form ultimately only decides what gets displayed or computed when you open a document. Adding or removing a field from a form doesn't add or remove an item from a document! A popular trap: You add a new field to a form (e.g. color) and give that field a default value (e.g. green). You open that document to check and it nicely shows "green". However the document actually doesn't contain the item. Since the item wasn't there when you open it, the default value executes, but until you switch to edit mode and hit save nothing gets saved back into the document.
The same applies when you remove a field from a document: it simply isn't shown anymore, but the data is still in the document. If you want to get rid of the data too, you need to create an agent for that (which is pretty simple: FIELD obsoleteField := @DeleteField (You get the irony here: despite deleting an item the formula is @Delete field. Renaming a field is the same as deleting and recreating it with a new name. It is an action in the form, with no impact on the document.
Another stumbling block: the form defines how an item is displayed and entered into a form via the field definition, suggesting data type and if it is one or more values. However items are always multi-value data and the data type of an item doesn't need to match the type of the field (if auto-conversion fails you will get an error). There is a detailed explanation about forms, documents and items you want to read.

Forms might include subforms, which from a document perspective doesn't matter. An item created (through a form) doesn't carry any information how it was created. If you need a detailed track record what item has been created/updated when and by whom, you want to have a look at the Inversion of Logging pattern.
Forms also contain a series of events that fire when things happen in the form. In the context of XPages that is no longer relevant, the event model is superseded by the XPages event model. However one area might be relevant: The @Formula used for fields:

* Keywords: check boxes, radio buttons, drop down list etc. Depending on type are inherently single or multi-value or both
Fields in forms can be computed, carrying one @Formula or Input fields, carrying 0-3 @Formula. A field defined as computed for display doesn't get stored back into the document. A field computed when composed gets computed once when the document doesn't contain the field yet. Usually that applies to new document, but the formula also would run when a field is added to a form and a document is opened with that form (and if the document isn't saved, the result of that formula doesn't get stored back). A computed field gets refreshed every time a document is recomputed, that implies the document is in edit mode (unless there is no item with that name, then it runs in view mode too)!
The translation formula replaces the user input with its result. A popular prank is to pick a formula that computes something independent from the input. For orderly developers there is @ThisValue to use the user input. The Validation formula needs to return @success or @Failure("message"). The failure message gets propagated to the XPages error system and shows in the form error control, but not in a field error control.
In XPages a developer can define for each document data source that the document should be computed on open and/or save. The difference to classic form use: in XPages on open all formulas run: default, translation and validation while in classic on open only default runs. Also the form computation runs after the XPages validations, so if an XPages validation fails, the form computation never runs which then might yield additional errors from the validation formula.
Next up: Not all Documents are created equally

Over the last five years I have trained many developers on XPages. A good portion of them came from a non-Notes development background: Java, dotNet, PHP and others. Most of them had a better grasp on the necessities of web development than most of their die-hard-is-there-anything-other-than-the-Notes-client LotusScript colleagues. However they struggled to fully appreciate the finer points of the NSF based nature of Domino development. The little article series is for them.
In short: Notes is a distributed application platform with an integrated NoSQL document database featuring a rich security and event model. It can be accessed in many different languages and protocols. While it is written in C, the most prominent languages used to develop applications in Notes are LotusScript (a Basic dialect, now de emphasised), JavaScript and Java. The most used protocols are HTTP(s), NRPC and SMTP.

In the beginning was the NSF

Almost everything in Notes is stored in NSF databases (NSF = Notes Storage Facility).

A NSF contains properties, an Access Control List, View indexes and a collection of Note elements, hence the name Note(s). The file name is not a property stored inside the NSF, however the title and the ReplicaID are. The ReplicaID has a special meaning: two databases sharing the same ReplicaID can synchronise their content in a process called replication, typically between a client and a server or multiple servers. When a server contains two or more NSF with the same replicaID replication can happen with any of them. A typical mistake: copy a NSF on the file system "as backup" creating not a copy but a replica. There is a menu action that creates a real copy for that (File - Application - New Copy).
Inside the NSF all data is stored in Note elements: both the design of the database, as well as the actual content. A data note (a.k.a Document) can be listed in a view (with the exception of a profile document), while a design note lists only in Domino Designer (unless you hack it). Design and data note elements share the same inner structure.
NSF do not contain any schema or data definitions. It is all Note and Item.

A Note is where you find anything

An individual note contains a series of properties and a collection of items (not to confuse with a field in an RDBMS or a field in a form). Besides data values for creation and update a note carries a UniversalId, a NoteId and a serial number. The NoteId is unique inside one specific NSF, the UniversalId is the identifier to link two note elements in two NSF (that have the same ReplicaId) together, so they synchronise on replication. The UniversalId is unique inside the NSF. Two different NSF can have documents with the same UniversalId. This is not an exception, but happens e.g. on mail routing. The serial number counts the times that note has been updated and is used in conflict resolution while replicating.

The contained items follow the same pattern: they have a name, a set of properties and a collection of values. Two important properties are name and type. The name of an item doesn't need to be unique. Quite regular, especially when looking at the type RichText you have more than one item with the same name (for RichText the unwritten convention is to use the name Body). Items that start with the $ sign are used for system properties (e.g. $HasNativeMime indicates that the note contains a MIME document rather than RichText). Items also contain a serial number for conflict resolution in replication. The values in an item share the same data type. Items are always multi-value (a Vector in Java) even if the count is just one.
Next up: Forms and Documents

XPages development uses well established standards: JSF, JavaScript, HTML, CSS, JSON, HTTP and XML.
However it is different from other web development environments:

• It is highly integrated. database, application and web tier come in one dense package. If your server is up Domino is good to go (no JDBC database timeout)
• The native database is NoSQL (actually the mother-of-all-noSQL), a document oriented store
• It is designed for distributed, partially connected operation
• Code can run on a server or on a client
• Domino provides build in directory and messaging capabilities
• It provides a hierarchical, declarative security model

To ease new developers into "how Notes is different" I will publish a series of articles, that focus on the big picture. They will make it easier to avoid " coding against the grain" and to appreciate the power of Domino. The plan so far:

1. The NSF
2. Forms and Documents
3. Not all Documents are created equally
4. Domino views are different
5. Finding data - Collections and Search
6. Better save than sorry - Security
7. Map Reduce Domino Style

Stay tuned

We all love our processes and the associated workflows. I recently even discovered a set of paper based ones at a customer site. I'm looking here at approval flows, not execution flows (that basically are checklists so everything is done in the right sequence and documented). In a nutshell they are all the same:

Someone request something, a set of approvers mused about it and the result has consequences. We all have build this type of applications in eMail, Notes, Sharepoint, dbBase, using spreadsheets, paper forms or high powered BPMN/ BPML/ BPEL engines. Workflow engines are supposed to ease the creation of the forms flowing through the process. They follow the same pattern: user fills in the form, some routing magic happens, the approver sees the same form, but with approve/reject and eventually a comment etc. We record who and when the approval happened (even using a signed section in Notes client apps) and the routing (and notification) magic kicks in again.
Since our systems are well designed and secure this works very well. Does it?
When we only record the who and when of approvals, but not the what, we open the door to the challenge:" I never approved THAT". So we need to capture a snapshot of how the record looked like at the moment of approval. Ideally that snapshot gets secured with a digital signature leading to non-repudiation. Now the next approver needs to not only endorse the data snapshot at the time of approval, but also the previous signature, so it can't be retracted either.

Now try to model that in an RDBMS (let me know if you succeed). This is one of the reasons why workflows are document oriented (sure you can persist it into an RDBMS, but you need to reassemble it to validate the signatures) and will stay that for the foreseeable future. The current "gold standard" for document signatures is XML Signature with an JSON equivalent in the making.
Some applications have support for signatures build in. For others we need to have a look at code. Stay tuned.

Update/Bonus challenge: Make the non-repudiation external verifiable (e.g. submit that to the court evidence collection). Hint: it is in the data, not the application

My Samsung Galaxy S4 decided more than two months ago to take a different flight. At the same time SWMBO declared that she had enough from her iPhone 5 and demanded a bigger screen. So I took the opportunity and switched to an iPhone (for now). Here is my completely biased, partial and unscientific - but 100% accurate review for the most important use case for me: my Smartphone usage:
I'm not much of a photography buff, player or mobile music listener. I check and answer emails, social content, get around and track activities. My experiences are based on that.

Topic	iPhone	Galaxy S4
Battery live	Lasts half a day with my usage pattern. In cycle mode: 2h screen operation	Lasts half a day with my usage pattern. In cycle mode: 2h screen operation
Screen	After having used a large screen is feels oddly small. I got big hands, so it can be larger. At the edged the screen touch precision is bad (planned obsolence?) Barly readable in sun light	Crisp, readable in most conditions, responsive, big
Keyboard	The biggest let down on the iPhone. I can't customize it. The caps lock is only visible on the caps lock button. Emoticons don't show up in Whatsapp. To get to the # I have 3 taps (unless the app requested the # like in Twitter). Auto-correct doesn't do multiple languages and I have to move fingers into the entry screen if I disagree. You can't navigate in text - try in a long URL that scrolled away to get to the front and change http to https - good luck with that.	I use Swiftkey that learned my vocabulary and also allows swyping. An ideal tool for frequent peckers. Wins hands down.
Home screen	Square icons, no spaces between them, max 9 icons per folder page: a joke when you had a customized Android home screen before. The swipe from top or bottom depending on the function is confusing. More than once, when I wanted to swipe up a website, article in Flipboard or Economist I ended up opening the control center.	Customization to my taste: my key applications (including IBM Notes Traveler) are configured as widgets, so I don't open them to glance on them (would be interesting to compare that with Windows phone) and there is space between my logical app groups
Music & Camera	iTunes and iPhoto make it easy to manage all the stuff	Music is a headache, camera works for my level (a.k.a Knipser)
eMail	gMail works similar on both devices. Apple mail shows its age. I can't swipe left/right for new messages but have to aim for the little arrows. I need a companion app for encrypted stuff	the Notes mail application works nice with swipe and support for encryption - and I can keep it separate from private messages
Contacts	One word: primitive	Allows to combine contact information from different sources into a single profile. And I can access my different services directly: Whatsup, Skype calling etc.
Accessories	Works quite nice with my Mio Alpha and Fitbit. There is also a bloodpressure monitor.	Until Android 4.4 hits my phones making low power bluetooth available, the Mio Alpha won't work. Fitbit works.
Quantified self	There is one aspect on the iPhone about Endomondo, haven't made up my mind if I like it: when starting the screen locks with a slider and keeps on, so I see it all the time. On Android it times out and locks with the passcode. Nice to watch on iPhone but a hug battery drain	I use Fitbit, Endomondo and WiThings. They work equally well
Control center	More than once when browsing a website or flipping upwards in Flipboard I end up pulling up the control center, unless the keyboard is displayed, then I can't pull it up at all. I like the flashlight.	The control center is in the main pull down, with all settings I need and an easy to target pull area
Tasks switching	iOS lets me switch back to the menu (single press) or the task list (double press). There is no concept of an universal back button, so when switching from one app to another no back brings me back to the previous, it is there sometimes depending on the app	The back button lets you backtrack nicely
Sharing	Apple knows best, so the share menu is what they tell me. I'm missing the IBM destinations (Connections, Sametime, Notes), Dropbox, UbuntuOne, WhatsUp, Evernote etc. Any anyhow, how can Apple know best if SWMBO already does?	Any application can register with the OS the capabilities it has to share things. Items appear in the Share via menu and live is good
Applications	All that I want to use can be found in the iTunes store	All that I want to use can be found, mostly in Google Play (some on my corporate server)
Development	ObjectiveC and Mac required. While I have a Mac at home, my workhorse is a Ubuntu Thinkpad	Java and any platform. My ObjectiveC is worse than my Mandarin (ask SWMBO how bad it is)

I most likely won't stick with an iPhone, while others want to ditch their Android
YMMV