AI Engineering Academy: 2.18Vision RAG Visual Capabilities

Claude Code is one of the most enjoyable AI Agent workflows to date. Not only does it make directed editing of code and improvised tool development less annoying, the experience of using it is even described as a pleasure in itself. It has enough autonomy to accomplish interesting tasks, while not giving developers the sudden...

When building knowledge base applications based on Retrieval Augmented Generation (RAG), document preprocessing and slicing (Chunking) is a key step to determine the final retrieval results. The open-source RAG engine RAGFlow provides a variety of slicing strategies, but its official documentation lacks clear explanations on the details of the methodology and specific cases, which brings a lot of confusion to developers....

When building Retrieval Augmented Generation (RAG) systems, developers often encounter the following perplexing scenarios: Headers of cross-page tables are left on the previous page, causing data to become unrelated. Models confidently give completely incorrect content in the face of ambiguous scans. The summation symbol “Σ” in a mathematical formula is incorrectly recognized as the letter “E”. Watermarks in documents...

Let's start with a simple task: scheduling a meeting. When a user says, “Hey, let's see if we can do a quick sync tomorrow?” An AI that relies only on Prompt Engineering might reply, “Yes, tomorrow is fine. What time would you like to schedule it, please?” This response, while correct, is mechanical and...

Abstract The emergence of large-scale language models (LLMs) has opened up a new paradigm of search engines that utilize generative models to gather and summarize information to answer user queries. We unify this emerging technology under the framework of generative engines (GEs), which generate accurate and personalized responses, rapidly replacing traditional search engines such as Google and ...

In the early days of the Manus project, the team faced a critical decision: should they train an end-to-end agent model based on open source models, or should they build agents that take advantage of the powerful “context learning” capabilities of cutting-edge models? Go back ten years and developers didn't even have a choice in the field of natural language processing. In the era of BERT, any model...

When building AI systems such as RAGs or AI agents, the quality of the retrieval is key in determining the upper limit of the system. Developers typically rely on two main retrieval techniques: keyword search and semantic search. Keyword search (e.g. BM25): fast and good at exact matching. However, once the wording of a user's question changes, the recall rate drops. ...

The experience of communicating with a friend who always forgets the content of the conversation and has to start from the beginning every time is undoubtedly inefficient and exhausting. However, this is precisely the norm for most current AI systems. They're powerful, but they're generally missing a key ingredient: memory. To build AI intelligences (Agents) that can truly learn, evolve, and collaborate, memory is not...