Introduction:

Following the seamless transition from QSTASH to a VPS, and with the application running smoothly, I received referrals from a satisfied client to three more potential clients. They expressed interest in having the same application with additional features.

Background:

This led me to launch three new containers, each with its subdomain for every client and interconnected via Caddy. However, this process unearthed more challenges than expected, particularly regarding resource allocation.

Challenges:

Cumbersome Email Scheduling:

Upon consulting with the new clients, I discovered they desired the ability to specify both the hour and minute for their emails. However, after delving deeper into their needs, I realized they sought to send emails at regular intervals. This prompted a revamp of the scheduling mechanism, transitioning from setting numerous timestamps to simply specifying a custom duration for email dispatch. This optimization not only streamlined the cron jobs but also enhanced performance and user experience.

Resource Management:

Each container operated its own Postgres database and cron jobs, resulting in redundant resource consumption. To address this, I transformed the system into a multi-user setup with a single container and a shared database, akin to modern SaaS applications. This consolidation significantly reduced resource wastage and improved cron scheduling efficiency.

Tailored Features for Each Client:

The transition to a SaaS-like architecture not only resolved resource inefficiencies but also simplified feature deployment. Instead of creating separate branches and custom code for individual clients, I adopted a unified approach, allowing simultaneous updates across all containers. This streamlined deployment procedures and enhanced agility in feature delivery.

Automation and Deployment Pain Points:

The manual update process, involving container shutdowns, code pulls, image rebuilding, and container restarts, proved laborious and prone to downtime. Recognizing the need for automation, I introduced Jenkins into the workflow. By configuring Jenkins to automate deployment tasks, such as pulling code changes and rebuilding containers, I alleviated deployment pains and enabled seamless feature integration.

Addressing Next.js Caching Challenges:

Next.js caching limitations posed a significant hurdle, particularly in a multi-client environment where data consistency was paramount. Transitioning to a SaaS architecture necessitated a shift in caching strategies, opting for client-side fetching with React Query to mitigate caching issues effectively.

Mitigating Gmail SMTP Constraints:

Utilizing Gmail SMTP for email dispatch posed significant challenges due to the imposed daily email quotas. Understanding the critical need for reliable email delivery while staying within these constraints, I engaged in extensive discussions with clients to devise a sustainable solution.

Collaboratively, we strategized to optimize email dispatch schedules, ensuring emails were sent efficiently while adhering to Gmail's limitations. After thorough deliberation, we agreed to truncate the 24-hour email dispatch window to a more manageable 9 hours, spanning from 7 AM to 4 PM.

Calculating the optimal email frequency within this timeframe, we settled on a duration of 3 minutes between each email dispatch. This meticulous planning ensured that the total number of emails sent per day per account remained well within Gmail's prescribed limits.

By implementing this refined scheduling strategy, clients were assured of uninterrupted email delivery to both their internal addresses for verification purposes and their target recipients. This collaborative approach not only addressed the immediate constraint posed by Gmail SMTP limitations but also fostered a deeper understanding of client needs and expectations.

Resolving Next.js Instrumentation and Cron Job Issues:

Technical glitches with Next.js instrumentation and cron job management necessitated a shift in approach. By decoupling cron-related functionalities from Next.js and implementing a separate Express.js server, I successfully resolved these issues and ensured reliable cron job scheduling.

Optimizing for Efficiency and Scalability:

Further optimizations included migrating the application from VPS to Vercel for cost efficiency and scalability, as well as transitioning the database to Neon DB for enhanced affordability.

Conclusion:

Through iterative problem-solving and adaptation, I successfully overcame various challenges in email scheduling, culminating in a robust and efficient solution. This journey underscores the importance of flexibility, automation, and client collaboration in delivering effective software solutions.