AI Assistant in Devopstrain

AI Assistant in Devopstrain 🤖

I'd like to share a rather interesting case study of using LLM AI models like GPT4-o.

We'll need the following ingredients:

➡️The model itself. I started with OpenAI GPT-4o, but their pricing went completely off the rails, so in the end I settled on Deepseek V3. The quality of the answers is at least no worse, and the price is 10 times lower.

➡️Course materials in text form. This was super easy, because my courses are built not around boring videos, but as interactive text containing both theory and practice. For this reason, feeding in the text materials turned out to be very simple.

➡️A platform for building scenarios that can request data from courses, handle basic logic, and craft the right prompts for the language model. This was admittedly a bit of overkill, but in return the universal platform now lets me create both Telegram assistants in the form of bots in a couple of clicks and assistants to embed on a website. And each of these assistants works according to its own scenario. There's actually a lot to say about this platform separately. For now it isn't publicly available, but if there's interest, I'll reveal the details in a separate post.

➡️A widget to integrate into the learning platform, in the form of a chat that you can see on the site and in the platform itself.

➡️ An adaptation layer to extract the text data from the JSON array that the platform returns. This is needed to save a lot on tokens, since the main expense is input tokens, which are required for "fine-tuning" the model. Roughly speaking, my largest section is about 13,000 tokens; without adaptation the model works great, but eats up about twice as much.

🧑‍🎓 You can see the result in my courses: you ask a question and get an answer not like a regular GPT, but one that takes the given course section into account. That's the main value and what sets it apart from a typical GPT 🤌