This is a submission for the AssemblyAI Challenge : Sophisticated Speech-to-Text.

What I Built

I built VoiceScribe, a cutting-edge speech-to-text application that leverages AssemblyAI's Universal-2 model to deliver precise, formatted, and highly contextual transcriptions. VoiceScribe is tailored to professionals and industries that require high-quality transcription with advanced features like proper noun recognition, timestamping, and seamless formatting.

Key features include:

High-Accuracy Transcriptions: Powered by Universal-2, VoiceScribe excels at converting complex audio into detailed and accurate text.
Automatic Formatting: Recognizes sentence structures, proper nouns, and numbers, ensuring polished results.
Timestamps for Context: Adds timestamps for every spoken segment to improve usability in meetings, interviews, and video editing workflows.
Searchable Archives: Enables keyword-based search within transcriptions for efficient information retrieval.
Multiple File Formats: Supports various audio formats, ensuring compatibility with diverse use cases.

Demo

https://sb1xsz849-zeh3--5173--d3acb9e1.local-credentialless.webcontainer.io/

Journey

To build this application, I integrated AssemblyAI’s Universal-2 Model API with a front-end built using React and a back-end using Node.js. Here’s how AssemblyAI enhanced my application:

Real-Time Speech Processing:
The Universal-2 model was critical for converting diverse audio inputs into text with high fidelity, even in noisy environments or with heavy accents.

Additional Prompts:

Summarization API: AssemblyAI’s summarization feature allowed me to generate concise outputs for meetings, interviews, and podcasts.
Topic Detection API: Incorporated to categorize audio into predefined topics, enhancing user experience for searching and organizing content.
Accessibility Features:
Leveraging AssemblyAI’s capabilities, I ensured that the app supports subtitles and closed captions for hearing-impaired users.

Challenges Faced
Noisy Audio: Mitigated transcription errors from background noise by preprocessing audio files.
Large File Sizes: Optimized file handling and API batching for long recordings.
Formatting Variability: Tuned the API integration to consistently produce human-readable text with minimal post-editing.
Future Plans
Adding real-time transcription for live events.
Integrating multilingual support for international users.
Developing mobile apps for transcription on the go.

This was a solo submission, but feedback and testing were conducted with [Teammate DEV Usernames, if any].

Thank you for the opportunity to participate in the AssemblyAI Challenge!