We're so proud of the work the OctoAI team is doing on MLCEngine – a Universal LLM Deployment Engine with ML Compilation 👏 Rebuilding an LLM engine that brings state of art serving optimizations, and maximum portability to diverse local environments. Learn all about it here and try it out! https://lnkd.in/edERBVvs
It is clear by now that the future of AI applications is hybrid, with model “cocktails” split between edge and cloud to offer new compelling AI experiences. The MLC-LLM project (and specifically the new MLC-Engine) is a fantastic colaborative effort to bring performance and portability across edge and cloud with a unified serving engine. It enables versatility in systems design to map right pieces of a model cocktail in the right place in the edge<>cloud spectrum. I am super proud of OctoAI being a leading contributor to this effort, along with Carnegie Mellon University and Paul G. Allen School of Computer Science & Engineering (all dear to my heart!) 😍 Learn all about it here and try it out! https://lnkd.in/gJDNfzir