SenseDeep DynamoDB Studio

Introducing the SenseDeep DynamoDB Studio.

SenseDeep has a complete DynamoDB developer studio to support your DynamoDB designs and single-table development.

It includes a table manager, data item browser, single-table designer, provisioning planner, database migration manager and in-depth table metrics.

SenseDeep DynamoDB studio is a comprehensive set of DynamoDB tools that are single-table "aware". This means SenseDeep can understand your single-table designs and application entity data and can guide your design, queries and monitoring based on this deeper understanding.

Background

DynamoDB best practices are evolving quickly as developers are realizing how to exploit the power behind the deceptively simple DynamoDB design. Designs with single-table design patterns, key overriding and composition, sparse indexes, query optimization and more powerful single-table access libraries such as OneTable are becoming common place.

However, managing single-table data and performance can often feel like you are peering at Assembly Language. Packing disparate data items into a single-table can make navigating, organizing and viewing data difficult. Furthermore, single-table design techniques such as prefixed and mapped attribute names exacerbate this problem and can make interpreting keys tough.

New tools are needed that understand the single-table schema and its relationships. These tools should support schema creating and be able to present and organize your data logically according to your application entities.

Data Browser

The SenseDeep data browser can query, manage and modify your table data. It supports browsing by scan, query or by single-table entities.

While the data browser can be used to browse and manage any DynamoDB table, it is turbo-charged when a schema describing the data is imported (such as from a OneTable schema) or defined using the SenseDeep single-table designer. When using a data schema, SenseDeep is able to "understand" your data and organize and present data items as application entities instead of raw, encoded DynamoDB items.

The SenseDeep data browser intelligently displays and formats data according to your single-table schema. This not only makes your encoded single-table data easier to understand, but it guides and validates your changes and prevents schema-breaking and application breaking data changes.

Queries

You can perform DynamoDB scans, native queries or queries by application entity.

When scanning, you select the index to scan and provide optional additional filtering attributes. Attributes can be combined in an AND or OR expression.

When querying, you specify the index and partition key with optional sort key value. Additional filters may also be specified.

When querying by Entity, you specify the index and application entity model name that is defined in the schema. SenseDeep then intelligently prompts you for the required attributes that comprise the keys to retrieve the data items.

Saving Queries

Queries may be saved to your database table where they are persisted in the schema. You can load and delete queries using the Queries button.

Data Visualization

SenseDeep groups, organizes and color-codes query results for maximum clarity.

Columns are ordered with the index partition and sort key first, followed by the schema entity type attribute (if defined). After that, columns for all items are ordered alphabetically. If an item has many attributes, click the edit icon to display the item attributes vertically.

Cells are color coded:

A cell is green if it can be edited. Click on the cell to modify the value.
A cell is pink if it has its value derived from a schema value template.
A cell has a wavy gray background if it is not relevant for this item (as defined by the schema).
A cell has a blue underline hot-link if it is a reference to another item and can be click to jump to the referenced item.

Hot Linked Items

If you control-click (or Cmd-Click) on a hot-link, you can quickly traverse related items. SenseDeep determines the relevant query to locate the item and automatically fills the index, key and filter attributes accordingly. You can save these queries using Queries -> Save. Queries are entered into your browser history so you can click the browser Back button to easily jump backwards to the original query.

Show Templates

A useful single-table design technique is to compose key values using templates that combine the values of other attributes at runtime to calculate the key values. It is often useful to view the value templates vs the calculated key values. The Templates toggle above the table switches between displaying the template values vs the actual data values.

Similarly, there may be some attributes that are designated as hidden. Changing the hidden toggle will display or hide these attributes.

Context Menu

On data cells, you can right click to display the context menu. The options are:

Copy to copy the current item to the clipboard in JSON format.
Design Schema will jump to the SenseDeep single-table designer for the entity on which the current item is based.
Edit will open the editor slide-out panel for easy editing of the whole item.
Follow Reference behaves the same as Cmd-Click to follow a hot link reference.
Generate Mock Data option can be used when developing to generate sample data, such as email addresses or phone numbers.

Modifying Data

You can modify data inline by clicking on any green cell. Once modified, the Save button will be displayed above the items to persist the changes to the table.

To add a new item, click the Add Item button. You can select the desired entity model (if a schema is present) and it will intelligently prompt you for the appropriate attributes.

You can also edit by clicking on the Edit pencil icon at the start of each item. This will display a slide out editor panel that will display only the attributes of the item, organized vertically. Click on a green cell to modify the contents. The red cells have their value derived from other attributes and cannot be edited. You can modify the value template in the schema via the SenseDeep single-table designer.

If you change the partition key values either directly or indirectly be changing attributes that are used in a key value template, SenseDeep will atomically remove the old item and create the new item via a DynamoDB transaction.

When you click save, the changes are accepted, but you still must click the Save button on the query page to persist results to the table.

Import / Export

No man is an island and your data must be easy to export or import. SenseDeep provides and Export to AWS Workbench option. This will export your schema and data items into a WorkBench model. This model can also be imported by the Dynobase tool.

When exporting, you should limit the amount of data you export. WorkBench models are for development and are designed for limited data sets. You can also export in a JSON backup format.

When importing, you can import a model to an existing (empty) table, or you can dynamically create a new table to hold the imported model.

Single Table Designer

The SenseDeep Single Table Designer provides an easy, intuitive interface to create, manage and modify your single-table schemas.

Single-table schemas control how your application data are stored in DynamoDB tables. Schemas define your application entities, table indexes and key parameters. Via schemas, complex mapped table items can be more clearly and reliably accessed, modified and presented.

The designer stores your single-table schemas in your table. In this manner, your table is self-describing as to how table data should be interpreted. You can export schemas and generate JSON, JavaScript or TypeScript data/code to import into your apps.

The SenseDeep single-table designer works best with the OneTable library, but should work with any consistent single-table design.

Naming and Versioning

Schemas are named and versioned. The default schema is called the Current schema. Other schemas can have any name of your choosing. For example "Prototype".

Schemas have version numbers so that data migrations can utilize the correct versioned schema when upgrading or downgrading data items in your table. The migration manager will select and apply the correct versioned schema when data migrations are run.

Version numbers use Semantic Versioning to indicate and control data compatibility.

Entity Models

An entity model defines the valid set of attributes for an application entity. Each attribute has a defined name, type and a set of properties that may be utilized by your DynamoDB access library such as OneTable or an ORM of your choosing.

From the list of entity models, you can click on an entity to modify, click Add Model to add a new entity model.

Via the Edit Schema button you can modify the schema name or version. If the schema is a non-current schema, you can also click Apply to Current to apply the contents of the displayed schema to the saved Current schema in the table. This overwrites the previous Current schema.

Entity Fields

Clicking on an entity attribute will display a slide out panel to edit the properties of that attribute.

You can modify the type and any other properties of the type. These properties are defined in the schema and may (or may not) be implemented by your DynamoDB access library. All these are implemented by OneTable.

The properties include:

Name — the name of the attribute.
Default Value — a default value to set if not defined when created.
Enumerated values — a list of possible values for the attribute.
Encrypt — hint the attribute should have an extra layer of encryption.
Filter — indicates the attribute can be used in filter expressions.
Hidden — the attribute should be hidden in query results by default.
Mapped Name — a shorter real attribute name when storing in the table.
Nulls — indicates that nulls should be stored in the table vs being removed.
Required — the attribute is required to be present when created.
Data Type — the attribute data type.
Validation Expression — a regular expression to validate all writes to the table.
Value template — a JavaScript value template to compose the attribute based on other attribute values.
Unique — indicates the attribute must always hold a unique value over all such items.

Export

You can also export your schema in JavaScript, TypeScript or JSON formats via the Export Schema button from the Models list. You can utilize the exported file directly in your OneTable apps to control your database interactions.

Provisioning Planner

The DynamoDB Provisioning page displays your current and projected provisioning, utilization and costs.

While there are various DynamoDB calculators, they are "theoretical" and not based on actual data usage. The SenseDeep provisioning planner displays displays the actual costs of your current billing plan based on real, live data. It then compares this usage with the the cost of an alternate plan if you were to switch your billing plan from OnDemand to Provisioned or vice-versa.

Database Migration Manager

The SenseDeep database migration manager provide a controlled way to upgrade or downgrade your table data.

It displays what migrations have been applied to your data and what migrations are outstanding. It shows what is the migration version for your current data and schema. You can upgrade by running outstanding migrations or downgrade by reversing migrations.

SenseDeep can orchestrate DynamoDB migrations controlled by the OneTable Migration library. You can use this library even if you are not using OneTable for your apps. Check out the OneTable Controller sample OneTable Controller in GitHub and deploy to host and manage your data migrations.

Read Configuring OneTable Migrate for more information and setup.

Table Metrics

SenseDeep provides both standard AWS DynamoDB metrics and enhanced single-table metrics.

The DynamoDB standard metrics displays overview metrics and operation details for a whole DynamoDB table. These metrics come from the standard AWS CloudWatch DynamoDB metrics.

DynamoDB Single-table designs are more complex and require per entity/model performance monitoring and metrics. Traditional monitoring covers table level and operation level metrics only. What is missing is the ability to see single-table entities and their performance and load on the database.

SenseDeep provides single-table metrics for apps using DynamoDB OneTable or for any JavaScript app that utilizes DynamoDB Metrics.

Table Management

SenseDeep can manage the DynamoDB tables for your enabled clouds. You can create and delete tables and indexes for your tables.

SenseDeep will automatically discover your tables and will dynamically update this list as new tables are created or destroyed. The table list includes the table size, number of items, billing scheme and provisioned capacity.

The SenseDeep table management is not meant to replace appropriate "infrastructure-as-code" deployment of tables to production. Rather, it intends to provide a quick and easy way to create and manage tables and indexes while developing your DynamoDB applications.

Summary

Gaining insight into single-table design patterns is the new frontier for DynamoDB and the SenseDeep DynamoDB Studio is the start of a new wave of tools to elevate and transform DynamoDB development.

Previously, single-table design with DynamoDB was a black box and it was difficult to peer inside and see how the components of your apps are operating and interacting. Now, SenseDeep can understand your data schema and can transform raw DynamoDB data to highlight your application entities and relationships and transform your effectiveness with DynamoDB.

SenseDeep includes a table manager, data item browser, single-table designer, provisioning planner, database migration manager and in-depth table metrics — all of which are single-table aware.

More?

Try the SenseDeep DynamoDB studio with a free developer license at SenseDeep App or learn more at https://www.sensedeep.com.

About SenseDeep

SenseDeep is an observability platform for AWS developers to accelerate the delivery and maintenance of serverless applications.

SenseDeep helps developers through the entire lifecycle to create observable, reliable and maintainable apps via an integrated serverless developer studio that includes deep insights into how your apps are performing.

To try SenseDeep, navigate your browser to: https://app.sensedeep.com.

To learn more about SenseDeep please see: https://www.sensedeep.com/product.

Please let us know what you think, we thrive on feedback. dev@sensedeep.com.

Blog