Partman Wizard - Partition tables from the command line

mateush

Matéush

Posted on August 28, 2024

Partman Wizard - Partition tables from the command line

This is a submission for the Neon Open Source Starter Kit Challenge : Ultimate Starter Kit

My Kit

Partman Wizard is a CLI tool designed to help you partition PostgreSQL tables using the pg_partman extension. This tool guides you through the process of setting up table partitions and migrating data efficiently.

What CLI can do?

  • Create and configure partitions based on existing tables
  • Migrate data from source tables to partitioned tables

...in one CLI tool.

Link to Kit

You can browse the source code for the Partman Wizard on GitHub. For a quick start, simply run the following command:

npx partman-wizard
Enter fullscreen mode Exit fullscreen mode

To see the tool in action, check out the video demonstration here:

The Journey

I'm Mateusz, a software engineer with experience across startups, SMEs, and large enterprises. I am in this game since more than a decade and I can tell that I have seen a lot.

Our company uses PostgreSQL in production, and one of our tables grew to over 10 million rows. Complex joins resulted in about 30 seconds of API response time (bottleneck was on the database side). The easiest solution to tackle this problem was either partitioning or materialized views. Materialized views weren't feasible since our SQL queries were always parametrized with user IDs. I found out about the pg_partman extension but unfortunately, our PostgreSQL provider didn’t support it. That means we needed to manage partitioning manually. What a shame.

A few days ago, I received an email from Dev.to saying there is a challenge related to PostgreSQL (Neon Open Source Starter Kit Challenge), and I thought it was the perfect opportunity to finally play with pg_partman. I decided to build Partman Wizard to automate the process of partitioning large PostgreSQL tables. Now you can do it from your command line.

PS. I am not sure if it meets your definition of a Starter Kit. I hope so.

Thank you for considering my submission!

💖 💪 🙅 🚩
mateush
Matéush

Posted on August 28, 2024

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related