Introducing GDPR-RAG

I've spent a lot of time thinking about Retrieval Augmented Generation (RAG). It is one of the best ways to make use of Large Language Models in a commercial application or environment. I've written a lot about them but now I have built one that I would like you to try so that I can get your thoughts and insight. 

Here is a service that answers questions on the General Data Protection Regulation (GDPR) based on the regulation itself plus some guidelines. If you are using a corporate device, your firewall rules may prevent you from seeing the application. You will know this is the case if you follow the link and see a screen that looks virtually empty, even after refreshing the page. You will know it works if you see a chat type screen. If it does not work on your corporate device, try your personal device or mobile. I would suggest talking to your IT department about changing the firewall rules, but we both know how that conversation will go.

Behind the scenes I have implemented most of what I have written about. In particular, I have taken a lot of time to chunk and "index" the documents which make up the corpus from which the service will answer questions. It will not be perfect as the index reflects my understanding of what a GDPR question is likely to be. Over time the index needs to be "fine-tuned" to actual user questions rather than relying on my best guess at what a user question may be. 

The service is the basic question answering version which means it is not intended to understand the conversation history. You need to pose complete, stand alone questions which the service will try to answer. If it can answer the question, it will provide references to the source material. I have written about this at length, one of the benefits of RAG over using pre-trained or fine tuned model is that you can answer with references. This also means that there will be more times where you can't get an answer. For commercial applications this makes more sense. Rather no answer than a well written, comprehensive and compelling wrong answer. 

There are many ways to improve a service like this over time. These include fine-tuning, (creating and) adding more documents to the index, adding Chat features rather than just Question Answering, and improving the interface. My hope here however is to convince you that you need something like this as a start in your corporation. RAG applications like this are ideal for Internal Support, User Manuals, Policies and Procedures, Workflows and any documentation that a new user to your corporation is likely to search for.

Please reach out to me and get some credentials. I would really appreciate your insights to see where best to spend my time to improve a service like this.