RAG in Production: Build a Document Q&A Bot with Qwen + Bailian

Everyone talks about RAG (retrieval-augmented generation). Fewer people show you the whole pipeline: how documents become chunks, how chunks become vectors, and how those vectors answer your users' questions. This post builds a production document Q&A bot with Qwen on Alibaba Cloud Bailian — the managed path that avoids running your own vector database.

Why managed RAG for most teams

• No vector database to operate (no pgvector tuning, no index rebuilds)
• Built-in document parsing: PDF, Word, Markdown handled out of the box
• Managed embeddings + retrieval + LLM in one API surface
• Scales without you thinking about it

The pipeline at a glance

Documents → Parse → Chunk → Embed → Vector Index
                                       ↓
User question → Embed → Retrieve top-k → LLM (Qwen) → Answer + citations

Step 1: Create the knowledge base

In Bailian, create a knowledge base and upload your documents. The console handles parsing and chunking. Practical tips:

• Chunk size around 500 tokens works well for most business documents
• Keep a small overlap between chunks so sentence context survives the split
• Name chunks with source metadata — you will need it for citations

Step 2: Wire retrieval + generation

With the knowledge base ID, the application code is short:

from dashscope import Application

app = Application(
    app_id=your_application_id,
    api_key=api_key
)

response = app.call(
    prompt="Summarize our refund policy in two sentences",
    rag_options={"pipeline_ids": [knowledge_base_id]}
)
print(response.output.text)

The response can include retrieved sources, which lets you render citations instead of hallucinating confidently.

Step 3: Production concerns

• Permissions: gate the bot behind your own auth layer; the knowledge base may contain sensitive docs
• Observability: log every question and its retrieved chunks — debugging a RAG app without this is painful
• Evaluation: build a small golden set of question/answer pairs and run it after every knowledge base update
• Cost control: cache common questions, use a smaller model for easy intents, and monitor token spend per session

Step 4: When to move off the managed path

Managed RAG is the right default. You only need to roll your own when you have very custom chunking, multi-tenant isolation at scale, or regulatory requirements around data residency that the managed service cannot meet. For those cases, the architecture is the same — you just own more of the pieces.

I keep independent notes on building AI applications with Qwen and current cloud pricing at lieke-ai.com. The official Bailian documentation and free-token campaign live here: Alibaba Cloud coupons.