LLM Agent Architecture for a Product Documentation Chatbot

An interesting variation from typical LLM application architecture

LLM is used twice:

· First for summarizing the conversation so far into a standalone question

· Then again with this standalone question and relevant docs for finding the answer

Though overall architecture follows the typical workflow:

· Mine data/documents, create embedding, and save in a vector db

· Construct prompts for user's queries along with matching docs retrieved from vector db

· Execute the prompt as an inference on pre-trained LLM