Implement Advanced RAG with Automated Query Rephrasing

This advanced workflow template builds a sophisticated RAG system that intelligently handles conversation history. It automatically reformats user follow-ups into standalone queries, then leverages OpenAI embeddings and Cohere reranking for ultra-precise knowledge retrieval. The result is a highly context-aware chatbot that delivers superior answer accuracy, significantly enhancing the user dialogue.

Workflow Overview

What is this?: This is a complete Dify chatflow that implements a sophisticated Retrieval-Augmented Generation (RAG) pipeline. It intelligently distinguishes between the start of a conversation and a follow-up question, using an LLM to rephrase subsequent queries for full context. The workflow then retrieves relevant information from your knowledge base using OpenAI embeddings and enhances accuracy with a Cohere reranking model.
What problem does it solve?: Are you frustrated when your chatbot loses track of the conversation after a few exchanges? This workflow solves that by giving your AI a 'memory', allowing it to understand follow-up questions like 'What about the second option?' by reformulating them into a complete, standalone query.
What will you get?: You will get a chatbot that can maintain a natural, multi-turn conversation. It dramatically reduces 'I don't understand' responses by always having the full context. Ultimately, you will create a more human-like and effective AI assistant for your users.

Apps Included

How to Use

Prerequisites

An active Dify instance (Cloud or self-hosted).

An OpenAI API key for the LLM and embedding models.

A Cohere API key for the reranking model.

At least one Knowledge Base configured in Dify with your documents.

Setup Steps

Import and Open the Workflow

Begin by installing this workflow into your Dify environment. Once installed, navigate to the workflow editor to view the complete node graph.

Configure API Credentials

Select the 'Query Reformatting LLM' and 'LLM' nodes and ensure your OpenAI API key is selected. In the 'KNOWLEDGE RETRIEVAL' node, verify that both your OpenAI key (for embeddings) and Cohere key (for reranking) are correctly configured.

Connect Your Knowledge Base

Click on the 'KNOWLEDGE RETRIEVAL' node. In its settings, select the specific Dify Knowledge Base you want this chatbot to search through.

Review the IF/ELSE Logic

Examine the 'IF/ELSE' node. It checks a conversation variable `is_beginning`. Notice how the 'true' path goes directly to the query aggregator, while the 'false' path first routes the query through the 'Query Reformatting LLM'.

Test the Conversation Flow

Publish and run the chat application. Ask an initial question, then ask a follow-up question (e.g., 'Tell me more about the first point.'). Observe how the chatbot correctly understands the context of your follow-up.

Pro Tips

Experiment with different reranking models in the 'KNOWLEDGE RETRIEVAL' node to see which provides the best performance for your specific dataset.

Consider adding a fallback branch for cases where the Knowledge Retrieval node finds no relevant documents, guiding the user to rephrase their question.

Back

Information

Published date8/15/2025
Last updated8/15/2025

Platform

Dify