Hey fellow devs! 👋

I've just published a comprehensive series on Medium detailing my journey of porting Phi-3-Vision, a powerful vision-language model, from Hugging Face to Apple's MLX framework. As a Python hobbyist, I wanted to share my experience and hopefully inspire others to dive into AI model optimization.

📚 Series Overview:

1. Basic Implementation: Getting Phi-3-Vision up and running in MLX.
2. Su-scaled Rotary Position Embeddings (SuRoPE): Implementing 128K context support.
3. Batching: Optimizing for multiple inputs.
4. Caching: Speeding up text generation.
5. Choice Selection: Implementing constrained output.
6. Constrained Decoding: Guiding the model's output structure.
7. LoRA Training: Fine-tuning the model efficiently.
8. Agent and Toolchain System: Building flexible AI workflows.

🤔 Why This Matters:

• Run advanced AI models efficiently on Apple Silicon
• Learn about model optimization techniques
• Understand the internals of vision-language models
• Explore the capabilities of MLX for AI development

🔗 Read the Full Series:

https://medium.com/@albersj66

💻 GitHub Repository:

I've open-sourced all the code and markdown files used in this series. You can find them in my GitHub repository:

https://github.com/JosefAlbers/Phi-3-Vision-MLX

Feel free to explore, experiment, and contribute!

💬 Let's Discuss:

• Have you worked with MLX or other AI frameworks on Apple Silicon?
• What challenges have you faced in porting or optimizing AI models?
• Any specific parts of the series you'd like to dive deeper into?

I'm excited to hear your thoughts and experiences! Let's learn from each other and push the boundaries of what's possible with AI on consumer hardware.