[Mar 2024] ML Community — Highlights and Achievements
Let’s explore highlights and accomplishments of the vast Google Machine Learning communities over the month. We appreciate all the activities and commitment by the community members. Without further ado, here are the key highlights!
Featured Stories
ML Developer’s Journey
AI/ML GDE Rubens Zimbres (Brazil) shared how he increased the number of readers and followers on his Medium channel. From Mar 2023 to Mar 2024 (1 year), readers have increased by 700% and followers increased by 800%! One of his articles, Augmenting Gemini-1.0-Pro with Knowledge Graphs via LangChain boosted his channel with reaching 2.9k reads in two weeks. (This tutorial gets 460+ Claps!👏) He attributed this increase in content popularity to 4 factors: 1) following and writing about cutting-edge technologies in Google Cloud, AI/ML and cyber security, 2) greater adoption of Google Cloud infrastructure to deliver AI/ML products, 3) Google for Developers’ social media support, 4) the platform’s support to writers with fresh and relevant material.
As an active AI/ML GDE and passionate writer, Rubens is constantly writing posts about ML technologies, his projects, and tutorials on Google’s latest ML products. Please check out his channel and enjoy his articles. And follow him!
Community Spotlight
The Machine Learning Singapore community led by AI/ML GDEs Sam Witteveen and Martin Andrews celebrated its 50th event 🎉 at Google Singapore office with talks/demos on Orchestrating LLMs and Agents with Gemini/Gemma.
Featured Activities
Gemma
Specializing Gemma models for your applications with Keras and LoRA by AI/ML GDE Vinicius F. Caridá (Brazil) was a workshop to simply apply Keras and Gemma to generate text responses to various prompts.
DSPy using Gemini and Gemma by AI/ML GDE Martin Andrews (Singapore) explained what the LMM orchestration framework DSPy is, and demonstrated various functionalities using Gemini (public version) and Gemma.
Gemma: from zero to hero (Colab Notebook) at TFUG São Paulo by AI/ML GDE Pedro Gengo (Brazil) and Lorenzo Cesconetto explained technical details about the model architecture and high level information. This session included demos of RAG using Gemma, Sentence Transformer and Faiss, fine-tuning Gemma using Keras, and running Gemma on llama.cpp.
Evaluating LLMs with LangChain: Using GPT-4 to Evaluate Google’s Open Model Gemma-2B-it by AI/ML GDE Rubens Zimbres (Brazil) analyzed 22 different metrics and found impressive results.
vLLM Faster LLM Inference || Gemma-2B and Camel-5B by AI/ML GDE Tarun R Jain (India) explains how to use vLLM, a fast and easy-to-use library for LLM inference and serving.
Using Gemma to generate data in five minutes by AI/ML GDE Jerry Wu (Taiwan) is a written tutorial to use Gemma to generate Chinese in 5 minutes.
Gemma for GST by AI/ML GDE Yogesh Kulkarni (India) explains how to fine-tune Gemma to generate a question answering model on Goods and Services Tax in India.
Built a Farmers bot using Intel oneAPI , Gemma-2B and Gemini Pro (repository | demo) by Usha Rengaraju (India) delves into the captivating endeavor of fine-tuning Gemma. This article got 370+ Claps in Medium.
Gemini
Multi-modal LLMs made easy: photo & video reasoning with Gemini 1.5 Pro by AI/ML GDE Gabriel Moreira (Brazil) demonstrated the multi-modal capabilities of Gemini 1.5 Pro by recreating his project, Photo Storytelling, which originally used PaLM 2 and Imagen APIs, but now using only AI Studio. With a simple prompt and images from his trip to Los Angeles attached, Gemini does a job creating a blog post describing the trip.
Ask Gemini about video clip in Python by AI/ML GDE Chansung Park (Korea) walks you through steps to ask questions about a video clip to Gemini 1.0 Pro Vision on Vertex AI.
Gemini AI and Python: My first app by AI/ML GDE Linda Lawton (US) guides how to use Gemini with Python to read your webcam and send multimodal requests to the Gemini Pro Vision. For example, she had the model read the webcam and tell the dollar bills that were displayed.
Episode 188 — Building Responsible AI with Gemini by AI/ML GDE Allen Firstenberg (US) delved into Gemini, specifically its built-in safety features designed to prevent harmful outputs like harassment, hate speech, sexually explicit content, and dangerous information.
Using Gemini 1.5 Pro to create video trailers by AI/ML GDE Dimitre Oliveira (Brazil) is about taking advantage of the Gemini’s multi-modal input to create trailers for any videos. It explores Gemini 1.5 Pro video processing power, discusses some experiments and applies it to a real use case.
End-to-End LLamaIndex and Gemini Project by AI/ML GDE Tarun R Jain (India) walks you through how to implement multimodal RAG project for Tourist Guide AI application using LlamaIndex, Gemini Pro and Qdrant.
Gemini Function and Vertex AI support in LangChain.js by AI/ML GDE Allen Firstenberg (US) are updates to LangChain.js to allow Function Calling support for AI Studio and Vertex AI APIs for Gemini. This package makes it easier for developers to get started with either platform.
Beyond Google Search: Unveiling Gemini for Javascript by AI/ML GDE Anshuman Mishra (India) explored Gemini, with a focus on its functionalities and accessibility for JavaScript developers. He delved into Gemini’s key features by showcasing its capabilities. The core of the talk was a step-by-step guide for JS developers, including how to leverage Gemini’s potential through GenerativeAI SDK.
Hands-on with Gemini by TFUG Agadir and AI/ML GDE Aashi Dutt (India) introduced Gemini and Gemma models with a demo of using Gemini 1.0 Pro and Gemini 1.5 in Colab, AI studio and Vertex AI.
Gemini: Hands-on Workshop by Deep Tech Stars covered practical introduction to Gemini especially about its capabilities, applications, and how to harness its power for your projects.
Colab
Colab 101: Your Ultimate Beginner’s Guide! by AI/ML GDE Sam Witteveen (Singapore) is a beginner’s guide to Colab introducing key features. This video got 2.4k+ views just in 4 days.
Gemma Fine-tuning Notebook by AI/ML GDE Merve Noyan (France) is a tutorial of fine-tuning Gemma with explanations on each step.
RAG with Gemma 2B by AI/ML GDE Pedro Lourenço (Brazil) is a Colab notebook on how to build a RAG pipeline using Gemma, Faiss, and Sentence-Transformer.
Unlock your ideas with Gemini: Hands-on guide by AI/ML GDE Nitin Tiwari (India) serves as a comprehensive hands-on guide to use Gemini in Colab notebooks, and demonstrates the capabilities of the Gemini 1.5 Pro in AI Studio. It also shows how Gemini Pro and Gemini Pro Vision can be used in Vertex AI on GCP to perform a wide variety of tasks.
Keras
Object Detection using KerasCV by Imen Masmoudi (Tunisia) explained the building blocks of the task and the transition between the public demo code shared by the Keras team that works on images.
Android implementation of the BERT model, developed through KerasNLP by AI/ML GDE George Soloupis (Greece) demonstrates the use of a .tflite model generated with KerasNLP within Android. It serves as a question-answering task where the goal is to pinpoint the exact span of text within the document housing the answer. The generated .tflite file can handle a sequence length of 512 input tokens making it appropriate to answer questions at almost a full A4 document. Use Gemini API inside android for a Question Answering task is his other article about an implementation of the Gemini API for the same task. The existing app was designed to support offline functionality, but the new app is finely tuned to seamlessly integrate both offline and online capabilities, allowing for a comprehensive comparison between the offline BERT model and the Gemini AI model.
Workshop: Build OCR Model for Reading Captchas with Keras 3.0 by TFUG Bassam and TFUG Abidjan guided how to create an optical character recognition (OCR) model that can read Captchas, the distorted images of text used for security verification.
Kaggle
ML Olympiads: detect hallucinations in LLMs with Google Gemini by AI/ML GDE Luca Massaron (Italy) talks about his ML Olympiad competition and how detecting hallucinations in LLMs is challenging and how Gemini and Vertex AI can help.
Fine Tuning Gemma LLM model (slides) by AI/ML GDE Tarun R Jain (India) at AICamp (Bangalore) discussed various topics including the utilization of Kaggle Models, techniques for parameter-efficient fine-tuning in LLMs, insights from the Gemma technical report, and conducted with a code demonstration.
Kaggle Notebooks (1, 2) for Home Credit — Credit Risk Model Stability competition by Usha Rengaraju (India) used Keras 3 to implement TabTransformer and Neural Decision forests.
On-device ML
Episode 185 — Cloud vs Local LLMs: A Developer’s Dilemma by AI/ML GDE Allen Firstenberg explores the rising trend of local LLMs, smaller models, such as Gemma, Mistral, Phi-2, Llama, designed to run on personal devices instead of relying on cloud-based APIs. Allen and Roger Kibbe discuss advantages and disadvantages of this approach, focusing on data privacy, control, cost efficiency, and the unique opportunities it presents for developers.
The power of Embeddings with Gemini and MediaPipe to find similarities in texts by AI/ML GDE Juan Guillermo Gomez Torres (Brazil) talked about the use of embeddings — to perform semantic search, clustering, recommendations, anomaly detection and classification, and etc. — and how to obtain them using Gemini and MediaPipe.
JAX
A new JAX Discord channel is launched in the Google Developer Community Discord server for JAX users and learners!
Mistral-7B reference implementation JAX and Equinox by AI/ML GDE Aakash Nain (India) is a repository containing a port of the original Mistral-7B model in JAX and Equinox. And he contributed to Equinox, a JAX-based framework, by adding dtypes with a simple test to make it easy to load LLMs onto it.
UvA DL tutorial series on “Training Models at Scale” by AI/ML GDE Phillip Lippe (Netherlands) explores parallelism strategies for training large deep learning models. The tutorial provides a comprehensive overview of techniques and strategies used for scaling deep learning models, and to provide a hands-on guide to implement these strategies from scratch in JAX with Flax using shard_map.
Activities by ML Frameworks
Building Knowledge Graphs from Scratch Using Neo4j and Vertex AI by AI/ML GDE Rubens Zimbres (Brazil) gives you the use of embeddings using Vertex AI instead of OpenAI, LangChain and graph visualizations in Neo4j Workspace. It is about knowledge graphs as a way to decrease hallucinations in LLMs via Cypher generation.
Generative AI and ML with Vertex AI by AI/ML GDE Ruqiya Bin Safi (Saudi Arabia) were at the Google Cloud booth at LEAP 2024. She was invited by GDG Cloud Saudi to host 2 sessions named Predict Visitor Purchases with a Classification Model in BigQuery ML, Generative AI and ML with Vertex AI.
Build powerful AI Chatbots with Google Vertex AI Conversation by AI/ML GDE Sachin Kumar (Qatar) is a video about how to build production-ready AI chatbots with Vertex AI Conversations. It explores what Vertex AI Search and Conversations are, how to create a datastore agent, and how to bring custom content from a website into an AI chatbot.
First Look: Duet AI in a Jupyter Notebook by Cloud GDE Lynn Langit (US) demonstrates Gen AI features in Duet AI using TensorFlow MNIST example using Vertex AI, Colab, and etc.
Building a Multimodal RAG with Gemini Pro and Gemini Pro Vision, Powered by Vertex AI by TFUG Islamabad delved into the creation of a multimodal RAG using Gemini Pro and Gemini Pro Vision, bolstered by Vertex AI. Moreover, the session emphasized the role of Vertex AI in providing scalable infrastructure for training and deploying such models.
