This is a submission for the AssemblyAI Challenge : Sophisticated Speech-to-Text.

What I Built

FluentAI is a cutting-edge AI-powered application designed to make English learning fun, engaging, and personalized. Whether you want to refine your reading, writing, or speaking skills, FluentAI offers tailored exercises, instant grading, and detailed feedback powered by advanced AI. It’s your one-stop solution to transform how you communicate in English!

How it works

Sign Up and Choose Your Focus

Start by signing up and creating your profile. Your details are securely stored in a PostgreSQL database for tracking your progress and storing feedback. Once logged in, select the area you want to work on: Reading, Writing, or Speaking.

Reading Practice

• Read aloud the given passage and record your audio.
• Once submitted, the audio file is processed:
  • Transcribed into text using AssemblyAI Universal.
  • Sent to Ollama Mistral Model for analysis.
• Receive scores and feedback on parameters like grammar, pronunciation, and tone of speech, all of which are updated in the database for future tracking.

Speaking Practice

• Choose a topic (or freestyle about your day!).
• Record and submit your audio.
• The same transcription and analysis process is applied to assess your fluency, clarity, and intonation.
• Personalized feedback and grades are stored in the database to help you chart your improvement.

Writing Practice

• Submit your written piece—be it an essay, story, or just a random thought.
• Your submission is sent to Ollama Mistral Model for a detailed evaluation based on parameters like structure, grammar, and coherence.
• Feedback is updated in the database to help you keep track of your writing progress.

PS: All your scores and feedback live safely in the database—no public displays of your grammar flubs or poetic masterpieces. 😅

Demo

checkout the live demo here:

fluentAi

Disclaimer: The responses may take around 5-10 min to generate as the server is deployed on AWS t2.large
(PS : I'm broke hehe..)

Landing Page

Reading section

Writing section

Speaking section

Feedback section

Source Code:

Journey

FluentAI was built to address a pressing challenge: providing an affordable, interactive way for users to improve their English communication skills. The integration of AssemblyAI’s Universal Speech-to-Text Model was crucial in bringing this vision to life.

AssemblyAI is the backbone of the audio transcription process, ensuring high accuracy and quick results.
The Ollama Mistral Model acts as the smart assistant, analyzing text and providing context-aware feedback.
It wasn’t just about building an app—it was about creating a seamless experience where users could learn, grow, and gain confidence in their abilities.

Tools Used

Building FluentAI involved a stack of powerful tools and technologies:

Frontend

• React: For crafting a dynamic and responsive user interface.
• Tailwind CSS: A utility-first framework to ensure sleek and modern designs.
• Shadcn: For reusable and aesthetic UI components.
• GSAP: For animations that make the app lively and engaging.

Backend

• Express + Node.js: For handling API requests and business logic.
• AssemblyAI: Converts audio into text with remarkable accuracy.
• Ollama Model (Mistral): Analyzes transcripts and text for personalized grading and feedback.

Infrastructure

• AWS EC2 (t2.large): For robust and scalable server hosting.(I'm Kinda broke, but this does the job..)
• Aiven Database: Stores user-profiles and progress securely.
• Vercel: Deploys the frontend for a fast and reliable user experience.

Final Thoughts

Building FluentAI has been an incredible journey, blending AI, creativity, and technology to address a real-world need. Whether you’re prepping for a presentation, working on essays, or simply polishing your conversational skills, FluentAI has your back.

With AI-driven grading and personalized feedback, learning English is no longer daunting—it’s a journey of constant growth and discovery.

Future Plans

I aim to make FluentAI even smarter with features like:

• Daily progress reports.
• AI-curated exercises based on your learning style.
• Gamified challenges to keep the learning fun and rewarding.