WORKSHOP

Dev, Sec & Ops meet Langchain

The emergence of LLMs and generative AI have added new exciting toys to our toolbelt for writing applications. Using code examples from the popular framework Langchain, I'll walk you through the concepts of this new domain and highlight dev & sec.& ops aspects along the way. With this, Patrick Debois, hopes to give you a peek under the hood and demystify the magic.

Lessons include, but not limited to:

Developers

• Calling a simple LLM using OpenAI
• Looking at debugging in Langchain
• Chatting with OpenAI as model
• Loading embeddings and documents in a vector database
• Show the use of OpenAI documentation to have the llm generate calls to find real-time information
  

Operations

• Find out how much tokens you are using and the cost
• How to cache your calls to an LLM using exact matching or embeddings
• Track your calls and log them to a file (using a callback handler)
• Impose output structure (as JSON) and have the LLM retry if it's not correct

Security

• Explain the OWASP top 10 for LLMS
• Show how simple prompt injection works and some mitigation strategies
• Check the answer llms provide and reflect if it ok
• Use a Hugging Face model to detect if an LLM output was toxic