AI Engineering Academy: 2.18Vision RAG Visual Capabilities

Let's start with a simple task: scheduling a meeting. When a user says, "Hey, let's see if we can do a quick sync tomorrow?" An AI that relies only on Prompt Engineering might reply, "Yes, tomorrow is fine. What time would you like to schedule it, please?" This response though...

Abstract The emergence of large-scale language models (LLMs) has opened up a new paradigm of search engines that utilize generative models to gather and summarize information to answer user queries. We unify this emerging technology under the framework of Generative Engines (GEs), which generate accurate and personalized responses that quickly ...

In the early days of the Manus project, the team faced a critical decision: should they train an end-to-end agent model based on open source models, or should they build agents that take advantage of the powerful "context learning" capabilities of cutting-edge models? Go back a decade and developers didn't even have a choice in the field of natural language processing. In ...

When building AI systems such as RAGs or AI agents, the quality of the retrieval is key in determining the upper limit of the system. Developers typically rely on two main retrieval techniques: keyword search and semantic search. Keyword search (e.g. BM25): fast and good at exact matching. But once a user asks a question worded...

The experience of communicating with a friend who always forgets the content of the conversation and has to start from the beginning every time is undoubtedly inefficient and exhausting. However, this is precisely the norm for most current AI systems. They're powerful, but they're generally missing a key ingredient: memory. To build systems that can truly learn, evolve, and collaborate...

From API calls for Large Language Models (LLMs) to autonomous, goal-driven Agentic Workflows, the paradigm for AI applications is undergoing a fundamental shift. The open source community has played a key role in this wave, spawning a plethora of AI focused on specific research tasks ...

Learn all about Reinforcement Learning (RL) and how to train your own DeepSeek-R1 inference model using Unsloth and GRPO. A complete guide from introductory to mastery. 🦥 What you'll learn What is RL? RLVR? PPO? GRPO? RLHF? RFT?...

With the rapid development and wide application of large-scale language modeling technology, its potential security risks have increasingly become the focus of the industry's attention. In order to address these challenges, many of the world's top technology companies, standardization organizations and research institutions have constructed and released their own security frameworks. In this paper, we will analyze nine of them...