In an organisation, while working with data, we tend to consider every new request as a new data pointer and expose it to the visualisation layer in an isolated manner. There isn’t a problem in most cases, however, what will happen when the organisation grows to a bigger scale? Problems will start surfacing out and causing issues. There will no longer be enough resources to sustain pulling larger data chunks, and there will be issues with data alignment across different data pulls or dashboards. The inevitable performance impact and increased computational cost of a query are better to be avoided, so let’s try to find out how.

This kind of problem requires retrospection as to how to gain leverage and have a data model or domain-driven design well coupled with a data model. To better understand how this is done, let’s see what domain-driven design actually is first. The “Domain” part in Domain-Driven Design officially refers to a “sphere of knowledge and activity around which the application logic revolves”. In other words, the “Domain” is what is commonly referred to as “business logic” in the software world. In Domain-Driven Design, business logic is considered to be the heart of the software.

In the Investment Banking world, for example, trade settlement is a domain where settlement ratio, settlement exposure, etc. are key metrics that are sliced based on Asset type or Account Type, etc. Regarding settlement as a domain is the first key towards adoption and anticipating the needs based on demand or past queries done.
Once you get to the depth of the domain, we can tie functional needs to an efficient scalable data model so that we can serve all data requests in a consistent manner, and in the best abstract way. This helps users to think about functional needs, and technologists work towards a scalable model as a repository to build.

Once the domain and modelling are in shape, then, irrespective of need, we should invoke one query with a different filter or dimensionality needed to serve it, along with the real-time or batch requests. At this stage, all abstract data models need to be exposed as a service/API (REST/SOAP), so that consumption can be done not only specific to visualisation or DB but for widespread use. This process of exposing makes your data go places while at the core of it is an efficient domain-driven design and a data model.

Some insight on how to build API/Service is to start with databases, queries as a collection in No SQL catalogue as configuration to map service, DAO to map service to API, ElastiCache to serve faster, Flask, Angular and Python for API and UI.

In conclusion, always look from a business lens to consolidate requests as a domain, build it into a model and serve as a batch or API in the service method.