DynamoDB Worx

DynamoDB is a serverless wide column key value store. It's a great database for Online Transaction Processing (OLTP).

I use this database a lot, and had been looking for a GUI to run locally. Those that I reviewed are either premium products, or open source options that are no longer supported.

The exception to that is NoSQL Workbench which is really good, but does not support my use case of needing to assume a role in another account after authenticating the source account with MFA (see here).

So I decided to build one, here we are 4 weeks later, and I would like to share my experiences with you.

Project Goals ¶

☑️ Multi account support
☑️ MFA Support
☑️ Support to Assume Role in another account
☑️ Edit the JSON documents using a text editor
☑️ Create, Update DynamoDB records
☑️ Perform Scans
☑️ Perform Queries
☑️ Support for DynamoDB local

Finished Product ¶

Check it out over here: https://github.com/simonireilly/dynamoDB-benchworx 👍

☑️ Creating/Updating Records ¶

The editor will detect changes to the primary key and sort key fields. This is a hint that the action will create or update a new record.

Basic usage profile selection and update/create

☑️ Query Planning ¶

Queries and scan are supported.

Below the planner you can see the parameters passed to the QueryCommandInput.

Querying on the primary key

☑️ MFA Support ¶

MFA support allows assuming roles in separate accounts after MFA auth 🥳.

Multi-Factor Authentication Prompt

Making the App ¶

This section is all about what went into the build, and how the DevOps have been set up.

The Stack ¶

Using electron so that I can publish to MacOS, Windows and Ubuntu 💻. I considered flutter, but I have zero experience so it wasn't the right choice for me.

Using React for the UI is just my defacto at this point...

Using typescript to write the code. This is something I think helps make code more legible and portable 👌

Using the AWS SDK for Javascript V3. This is something new I wanted to learn, so a good addition to the mix for me 🤩

Finally for all DevOps using GitHub to collaborate, build, test, deploy and receive feedback (issues).

The Sub Stack ¶

Early on I decided to rely on a User Interface kit, this would help me go faster on the MVP, so the amazing Material UI Core was used.

For an editor, I personally enjoy VSCode, and so, a massive benefit was the @monaco-editor/react, which dropped in nicely. I hope to enhance the app using its diff functionality soon too!

The process ¶

I was aware of electron-forge and electron-builder from previous apps, and so, began with a bootstrapped sample from electron-forge.

I followed the steps for the Electron forge with webpack and Typescript example on electron forge.

Electrons Sharp Edges ¶

The main sticking point that can trip you up in electron is the concept of renderer and main.

Main is the process that is running your application. This main process is responsible for booting, preloading and preparing your application along with any node_modules that you require.

Renderer processes are detached from the main process, and communicate over the IPC bus in electron. This has great security and segregation wins. It enables the launching of multiple windows and keeping windows secure.

Previously I had found the whole concept of forwarding dependencies to the renderer process too tricky. Electron has improved this since I used it in 2019, and now it includes a method to append functions onto the window object. The function is called exposeInMainWorld 👏.

This is a great name, as this is exactly what the function does. It enables you to overload the renderers Window object with additional functions that you would like to call. These functions can be from node_modules as long as they are imported in the preloader.ts. Main handles preloading these and ensures that when the first renderer is opened, all the preloads are accessible on the window.

An example of preloading is below:

// ../_snippets/dynamodb-worx/preloader.ts

// Preloading allows fetching node_modules/native code into the application
//
// In order to use the @aws-sdk we must import those utilities here and add
// those methods to the window.
//
// When writing those methods, we should encapsulate the actual calls to client,
// preventing injection and/or privilege escalation.

console.info("Preloading node modules");

import { contextBridge } from "electron";
import { isDeepStrictEqual } from "util";

import {
  describeTable,
  listTables,
  scan,
  query,
  put,
} from "@src/utils/aws/dynamo/queries";
import { listAwsConfig } from "@src/utils/aws/accounts/config";
import { authenticator } from "@src/utils/aws/credentials";

declare global {
  interface Window {
    aws: {
      authenticator: typeof authenticator;
      describeTable: typeof describeTable;
      listAwsConfig: typeof listAwsConfig;
      listTables: typeof listTables;
      scan: typeof scan;
      query: typeof query;
      put: typeof put;
    };
    util: {
      isDeepStrictEqual: typeof isDeepStrictEqual;
    };
  }
}

// All responses should implement this interface
export type PreloaderResponse<T> = {
  type: "success" | "error" | "warning" | "info";
  data: T | null;
  message: string;
  details: string | null;
};

// Expose the AWS object to over the context bridge to allow invocation of the attached
// methods in the renderer processes.
contextBridge.exposeInMainWorld("aws", {
  authenticator,
  describeTable,
  listAwsConfig,
  listTables,
  scan,
  query,
  put,
});

contextBridge.exposeInMainWorld("util", {
  isDeepStrictEqual,
});

¶

The DevOps ¶

I identified that I wanted:

Visual test automation
Unit test automation
Automated releases

Visual test Automation ¶

I really like vercel's screenshots for new releases as it helps me iterate quickly on my web apps.

I think its important to also recognize the duality of modern front end. While front ends are predominantly visual, modern javascript fat clients have placed a barrier between the designer and end product. I had hoped these review comments would make it possible to qualify the design implementation without needing to get into the code base.

I quickly put something together for this using:

The github action I created would use the github token of the user who submits the pull request to add a snapshot comment. Because of this I had to exclude Dependabot. The token Dependabot uses has less permissions.

# ../_snippets/dynamodb-worx/ci.yml

name: Lint/Test/Build

on:
  push:

jobs:
  screenshot:
    - name: Append current screenshot
      uses: peter-evans/commit-comment@v1
      if: github.actor != 'dependabot[bot]' && github.actor != 'dependabot-preview[bot]'
      with:
        body: |
          ### 📷 Snapshot Tests

          | Previous UI | Current UI |
          | ----------- | ---------- |
          | ![Current snapshot](https://raw.githubusercontent.com/simonireilly/dynamoDB-benchworx/main/cypress/snapshots/end-to-end/index.spec.tsx/latest.snap.png) | ![Current snapshot](https://raw.githubusercontent.com/simonireilly/dynamoDB-benchworx/$/cypress/snapshots/end-to-end/index.spec.tsx/latest.snap.png) |

          Changes to the UI that are applicable after this merge request.

The outcome was exactly what I was after:

Screenshot diffing comments on GitHub

Unit Test Automation ¶

Simple unit testing has been more than enough, and for this, Jest is still my preferred runner. My methodology was to test the AWS SDK calls used by my custom context object to ensure they returned the Preloader interfaces defined above.

This ensures that the front end will display a notification of the error. I don't like to obscure API calls behind custom interfaces, so there was a light wrapper that mimics axios/fetch for response.type === 'success' assertions. Errors are logged into the console as well, so users can debug. I assumed most users will be developers, and that my code will break, so if they can debug it I think it will prevent a lot of frustration.

Automated releases ¶

I wanted a low maintenance release process that would work 100% in GitHub.

I ended up using a combination of:

Electron Forge Github Publisher - To create binaries and assets
Release It - To create tags, bump the versions, manage release betas and detect merged changes.
Release It Auto Changelog - To automate the creation of a CHANGELOG.md

I use release-it locally to create the release, then once that is pushed to a tag, a GitHub action picks it up and adds the compiled assets for MacOS, Windows and Ubuntu.

The github action for publishing is really straight forward thanks to Electron Forge

# ../_snippets/dynamodb-worx/release.yml

name: Release

on:
  push:
    keywords:
      - "v*"

jobs:
  build:
    runs-on: $

    strategy:
      matrix:
        os: [macos-latest, ubuntu-latest, windows-latest]

    steps:
      - name: Check out Git repository
        uses: actions/checkout@v1

      - name: Install packages
        run: yarn install

      - name: On Publish
        env:
          GITHUB_TOKEN: $
        run: yarn electron-forge publish

The end result is pretty satisfying, and I am really happy that this process is almost entirely automated.

Automated releases showing the changelog and binaries

Going Forward ¶

There are a few features that I cut from the MVP and I will be working on soon:

Code signing for Windows and MacOS, this is a big one and for sure this is something I will need to learn about before implementing.
Deleting Records.
Create Read Update Delete (CRUD) for tables.
Remote to local replication, of table schemas, to create local throw-away copies for designing complex queries.
Two way binding of the Query Builder forms and a editor to allow entering AWS SDK V3 DynamoDB commands without using forms.

And probably many more things!!

Wrap Up ¶

Thanks for reading and if you want to learn more please check out the project here: https://github.com/simonireilly/dynamoDB-benchworx