When building something similar powered by OpenAI I had a real pain in the ass anonymizing the data, then de-anonymizing the answers before showing it to the customer.
Also in my example, I'm sure using a string like "Pineapple Cave Inc." instead of the real business name hurt the AI's ability to contextualize the information and data and that hurt the LLM somewhat -- right?