Requirement Analysis - Rules vs Lexicons

Intro

How do you call yourself? Software developer, software engineer, coder, programmer, consultant, project manager, etc.?

And how much do you get involved with Requirement Analysis? That would heavily depend on what your title is, and the policies of the corporate you work in or the culture of the communities you belong to.

You may just need to receive the requirements as the artifacts completed by professional requirement analysts or domain experts. Your job is to implement exactly what have been specified and documented, no more, no less.

Or you may get heavily involved in it, questioning every bit of details in the requirements to ensure that later these requirements are feasible to implement. You act as a requirement analyst, documenting the comprehensive specifications for the requirements.

Or you may be in the middle of the two above, working on the raw and unrefined requirements collected by your consultants or your primary users. You support your project stakeholders in documenting the requirements, and your team as a whole act as a requirement analyst team (as it is the most complex profession that are not always available within 1 person).

Whatever case you are in, there are one fact that never changes:

You will know whether the requirements are good or bad, sooner or later.

Your experience and intuition sometimes are able to tell the sign early.

However, it is usually difficult to comprehend what your experience or intuition says, so even seasoned consultants and senior programmers can sometimes find themselves in troubles to explain why they think certain requirements are bad and need to be refined more before implemented.

We all want to tell and define what good and bad requirements should look like, as early as possible. So, this article (or post) here will discuss an aspect that in hope can help you figure out one aspect in analyzing the requirements.

The Game of Life

Anyone knows about Conway's Game of Life?

If you already know it, let's pretend you do not know about its 4 Rules yet.

Otherwise, if this is the first time you hear about it, here is the first recap:

You are given a 2D grid of square cells, each has either dead ⬜ or live 🟩 states.
You can toggle to change the cells' states. All cells are dead ⬜ by default.
You can then leave the cells to change their states over time. At each step of time, how the cells change or remain their states follows the 4 Rules.

You can find the online version of Conway's Game of Life here. But before you go there, try not to look at the "Explanation" on the 4 Rules.

If you visit the online version site, you will likely see a pattern that is setup as the default (if it is not, try set it up):

⬜🟩⬜
⬜⬜🟩
🟩🟩🟩

You can proceed to see how cells change over time by pressing "Start". You will find live cells of this pattern move south-east.

This is one of the Lexicons, called the Glider.

Requirements by Lexicons

What are the Lexicons? They are simply presets of patterns, mostly common ones.

Now, imagine you are requested, as a coder, to re-create the Game of Life. You do not know the 4 Rules that dictate the Game of Life. Instead, you are briefed with the following observations of 5 Lexicons:

1: If you input the pattern of single isolated live cell, the cell will die in the next step of time:

From:
⬜⬜⬜
⬜🟩⬜
⬜⬜⬜

To:
⬜⬜⬜
⬜⬜⬜
⬜⬜⬜

2: The same is applied to 2 live cells living next to each other:

From:
⬜⬜⬜⬜
⬜🟩🟩⬜
⬜⬜⬜⬜

To:
⬜⬜⬜⬜
⬜⬜⬜⬜
⬜⬜⬜⬜

3: For 4 live cells that creates a shape of a 2x2 square, they will not die:

⬜⬜⬜⬜
⬜🟩🟩⬜
⬜🟩🟩⬜
⬜⬜⬜⬜

4: If there is one cell missing in the 2x2 square, the missing cell will be refilled:

From:
⬜⬜⬜⬜
⬜🟩⬜⬜
⬜🟩🟩⬜
⬜⬜⬜⬜

To:
⬜⬜⬜⬜
⬜🟩🟩⬜
⬜🟩🟩⬜
⬜⬜⬜⬜

5: If there are 3 live cells in 1 line, the line will rotate 90 degrees in the next time step:

From:
⬜⬜⬜⬜⬜
⬜⬜⬜⬜⬜
⬜🟩🟩🟩⬜
⬜⬜⬜⬜⬜
⬜⬜⬜⬜⬜

To:
⬜⬜⬜⬜⬜
⬜⬜🟩⬜⬜
⬜⬜🟩⬜⬜
⬜⬜🟩⬜⬜
⬜⬜⬜⬜⬜

You are asked to implement the Game of Life by these 5 Lexicons. Your code may look like this:

...
if (one_cell_pattern) or (two_cells_pattern):
    die;
if (square_of_two_pattern):
    no_change;
if (square_of_two_missing_one_pattern):
    change_to(square_of_two_pattern);
if (3_cell_line):
    rotate(90);
...

Can you see the problems now?

Your code will increase its complexity over time when there are new Lexicons introduced to your Game of Life.
You may be promised that the players won't input any other Lexicons that are not specified in your requirements. However, that also means your Game of Life have to validate whether the players input any unspecified Lexicons.
You may be promised that the players won't input Lexicons that can collide each other. However, that also means your Game of Life have to validate whether the players input Lexicons that should be far enough from each other (at least 2 dead cells apart). Try this Lexicon for example on the online Game of Life:

⬜⬜⬜⬜⬜⬜⬜
⬜🟩⬜⬜⬜🟩⬜
⬜🟩⬜🟩⬜🟩⬜
⬜🟩⬜⬜⬜🟩⬜
⬜⬜⬜⬜⬜⬜⬜

You are hoping the players won't know about the Glider Lexicon and request to include it into the Game of Life...

Requirements by Rules

OK, now the same situation where you are to implement Game of Life again. But instead of implementing by Lexicons, you are about to implement by the 4 Rules.

Here are the 4 Rules of Game of Life:

Every cell interacts with its 8 neighbours, which are the cells that are horizontally, vertically, or diagonally adjacent. At each step in time, the following transitions occur:

1: Any live cell with fewer than 2 live neighbours dies, as if by underpopulation. Example:

From:
⬜⬜⬜
⬜🟩⬜
⬜⬜⬜

To:
⬜⬜⬜
⬜⬜⬜
⬜⬜⬜

2: Any live cell with 2 or 3 live neighbours lives on to the next generation. Example:

⬜⬜⬜⬜
⬜🟩🟩⬜
⬜🟩🟩⬜
⬜⬜⬜⬜

⬜⬜⬜⬜⬜
⬜⬜🟩⬜⬜
⬜🟩⬜🟩⬜
⬜⬜🟩⬜⬜
⬜⬜⬜⬜⬜

3: Any dead cell with exactly 3 live neighbours becomes a live cell, as if by reproduction.

From:
⬜⬜⬜⬜
⬜🟩⬜⬜
⬜🟩🟩⬜
⬜⬜⬜⬜

To:
⬜⬜⬜⬜
⬜🟩🟩⬜
⬜🟩🟩⬜
⬜⬜⬜⬜

4: Any live cell with more than 3 live neighbours dies, as if by overpopulation. Example:

From:
⬜⬜⬜⬜⬜
⬜🟩⬜🟩⬜
⬜⬜🟩⬜⬜
⬜🟩⬜🟩⬜
⬜⬜⬜⬜⬜

To:
⬜⬜⬜⬜⬜
⬜⬜🟩⬜⬜
⬜🟩⬜🟩⬜
⬜⬜🟩⬜⬜
⬜⬜⬜⬜⬜

Now you just need implement the 4 Rules, then all possible Lexicons will follow and manifest by themselves.

So what? My requirements are not for the Game of Life

You are right. But the point of this example is to show you 2 types of Requirements that you will encounter in reality. I will call them by the names in the Game of Life:

Rules: They are mostly small, simple, and sometimes boring. A Rule, once stated, never have an exception. Rules are designed not to conflict or collide with each other. In the Game of Life, the 4 Rules are isolated from each other based on state of the cell (dead or live) and the number of live neighbors around that cell.
Lexicons: They are phenomena created as the products of Rules. Some are fascinating (remember the Glider). They usually don't have exception by themselves, but they can conflict or collide with each other to create unexpected phenomena.

Problem 1: Blinded by Lexicons

The Lexicons of any business domain are the reasons why people use the system. Therefore, they are mistaken as the most valuable part of any software product. Most of the time, when a software product is getting published or distributed, stakeholders focus on testing whether its Lexicons are in place:

If I dine in this restaurant, my total bill will have an additional charge of 18.8%.
But if I take away, the additional charge should be a fixed 30 cents per order, and 8% overall including the fixed charges.
The compensation of a half-day event should be $100.
But if the event is full-day event, it should be $250.

Problem 2: Programmers are too far away to be aware of the Rules

As stakeholders are mostly blinded by the Lexicons, the requirements when reaching programmers can be described only in the terms of Lexicons.

Furthermore, the stakeholders who should know the Rules the most are the end-users or the domain experts, not the programmers. However, as Rules are mostly small, simple, and insignificant, these stakeholders may fail to describe the Rules to other stakeholders in early time. It's similar to that we find it difficult to describe how to breath.

Problem 3: Implementing by Lexicons is inevitable

As stakeholders who are closer to the implementation side are not likely aware of the actual Rules of the system they are implementing, it is inevitable that they would implement based on Lexicons first, especially during the early time.

Of course, that does not mean that any software development product would be implemented by Lexicons all the time. Stakeholders, especially programmers, tend to recognize the conflicts within the Lexicons, and if they are determined to resolve these conflicts, they can unveil the unconflicted Rules behind them, which they would implement the product based on the Rules. For examples, the Rules of above Lexicons can be:

This restaurant when dining in will charge 10% of sub-total (total prices of all orders). Otherwise, it will charge 30 cents as containers for each order.
This restaurant has to charge additional 8% GST for all.
Rate of event compensation is 25$ per hour.
Any half-day event is either 8am-12am or 2pm-6pm. Full-day event has to cover from 8am to 6pm.

Problem 4: It may be difficult to shift from Lexicons to Rules of an existing software

Even when stakeholders find out the Rules they should implement, they may find it difficult to rebuild on the Rules, depending on how the software product is architected.

The most typical case is stakeholders refuse to revise the implementation from Lexicons to Rules for fear that the Lexicons may fail to hold when the implementation changes. This case is likely to happen to projects that lack of the test suite around the Lexicons, or the test suite is so highly coupled with the Lexicons' implementations (not behaviors) that to replace them with the Rules implementation, a majority of the test suite have to be removed.

Outro

So, what can we do from these problems? If you have read to this point of this article, congratulations as you already get to the first solution.

Solution 1: Awareness of the existence of Rules and Lexicons of the domains

What you thought as the Rules of the domain your software is implementing may be the Lexicons. The easiest red flag is when you see possibilities of such "Rules" conflicting with each other.

Another red flag is that over the time of development or usage, the "Rules" that are thought to be constant are requested to be amended, either as altered, removed, or complimented with new "Rules."

Solution 2: Detecting as much Rules as early as possible

Highly skilled domain experts or business analysts can detect and filter out the Rules from the Lexicons. They will use the Rules as specifications of the requirement, while listing the Lexicons as the resulting examples of the Rules.

However, not all projects are lucky to uproot all the Rules early on, even if they have decent requirement analysts. Therefore, to be more practical, projects should try to at least identify and reserve strong and simple Rules that cannot conflict with each other and cannot be changed. That should lay the ground for the next solution.

Solution 3: Architect to allow the shift from Lexicons to Rules easier in the future

Implementation by Lexicons cannot be avoided, with various degrees. Therefore, software development team can place some measures to allow themselves to implement by Lexicons first, and later to shift to Rules without much hassle. For examples:

Architect such that the implementations over Lexicons are easy to change later. Following the SOLID principles is an example.
Strong test suite around what are identified as Rules' implementations.
Weak or complimenting test suite around the Lexicons' implementations. If possible, make it that test suite around the Lexicons is not necessary, where the test suite around the Rules should be sufficient. Otherwise, consider isolating the Lexicon test suite from the Rule test suite. The end result is that you would want to minimize the cost to alter the test suite when the Lexicons' implementation changes.

Blog