Not something that I have tested but you likely can do this with Ollama and probably some python code.
Just saw, it’s got a free trial period and a one of fee of only 50 bucks
If it does what it says on the tin, respects privacy, with nothing sent back to the mothership, with no hidden charges, sounds like a ruddy bargain 
Giving it a go, cynic in me says NO!
Sounds like just an indexing service with the smallest amount of inference. But if it makes your life better, probably worth it. Let us know how well it does or does not work.
My needs are not that detailed anymore, it would just be nice to point it to literally years and years of stored info….
if you want to do something similar, you can feed your documents to an embed llm and store the vector result in a database of your choosing.
Just make sure you use the same llm model for the prompt (query) or your coordinates may not entirely align and get flukes.
huh i tried a simple setup with an open source embed model and stored the vectors in qdrant it worked fine for a few hundred docs edit you can also use milvus if you need scale but it adds some overhead