A developer's guide to Large Language Models

So your manager has drunk the Kool-Aid and wants an LLM of their own. Of course, they're not going to spend millions of dollars building a team, renting a GPU farm, and hiring specialists to tailor bespoke training data. No, they have you, and as a side project that definitely must not interfere with your day job, you need to build them something a little like GPT-4 - only better because it must also be able to answer questions about their internal documentation.

If that sounds like you, you are in the right place and here you are among friends.

In this series, I try to answer the question: How do I get a Large Language Model to answer questions on documentation that was not part of its training set?

Without spoiling the ride, let me just say that you must have been a very good little developer because not only is there an answer to this question but the answer also happens to coincide with the approach that is the least difficult option to put into production.