AI Chatbot Architecture with Llama 3.1 and GPT-4o-mini

ML & AI · flowchart diagram · NOASSERTION

Illustrates an AI chatbot's RAG architecture, detailing how user queries are processed through summarization, a supervisor LLM, vectorstore retrieval, and

Source: https://github.com/eCom-dev5/verta-chatbot/blob/91f0c62187b6a69db2dcbe5b0259449b1cc43dd4/readme/01_BASE_MODEL.md
Curated by eCom-dev5
AI Chatbot LLM RAG Llama 3.1 GPT-4o-mini LangFuse Vectorstore

Mermaid source

flowchart TD
    start[User Query]
    start --> summarizer["Metadata Summarizer (Llama 3.1 8b)"]
    summarizer["Metadata Summarizer (Llama 3.1 8B)"] --> supervisor["Supervisor (Gpt-4o-mini)"]
    supervisor -->|Unstructured Query| vectorstore["Vectorstore Retriever (FAISS dB)"]
    supervisor -->|General Query| mainllm["Main LLM (LLaMA 3.1 70B)"]
    vectorstore --> supervisor
    mainllm --> followup["Follow-up Question Generator (Llama 3.1 70B)"]
    followup --> response["Response to User"]
    response --> langfuse["LangFuse Analytics"]

What this diagram shows

This diagram illustrates the architectural workflow of an AI chatbot. It details the journey of a user query, starting with a metadata summarizer (Llama 3.1 8B), moving through a supervisor LLM (GPT-4o-mini) that directs queries to either a vectorstore retriever (FAISS dB) or a main LLM (Llama 3.1 70B), and finally generating a response with follow-up questions, all while being monitored by LangFuse analytics.

When to use it

Use this architecture for building advanced AI chatbots that require intelligent routing of queries, retrieval-augmented generation (RAG) for specific knowledge bases, and robust performance monitoring. It's suitable for applications needing to handle both structured and unstructured data queries efficiently.

How to adapt it for your project

Adapt this by swapping LLM models, integrating different vector databases, adding more specialized agents for specific query types, or incorporating additional monitoring tools. The summarizer and supervisor roles can be fine-tuned for specific domain knowledge or expanded to include more complex decision-making logic.

Key concepts