Hey, I’m currently trying to get away from Notion to either Obsidian or Logseq. Currently it feels like I’m drifting towards Obsidian.
I only have one topic left on my list of “I would like to have that”: an alternative to Notion AI.
With Notion AI it’s possible to ask a question like “which books that I’ve read are talking about the history of mankind and how are these related to each other?” and it’ll search your documents and give you an answer. This works really good. Downside of Notion is that you don’t earn your files and that of course the AI model is scanning all your stuff for this when you use it.
I’ve seen that there are some add-ons for Obsidian that support AI but I haven’t really achieved anything good yet. My try was to set up Ollama on my Linux computer and connect it to some Obsidian plugin (not sure about the name right now unfortunately). As a model I was using Llama3 (the small one since the big one is too heavy for my laptop with NVIDIA GPU).
That in fact works but the results are…meh. It kinda leaves out 80% of what would be relevant and thus isn’t really helpful.
Is it somehow possible to achieve something like an alternative to Notion AI in Obsidian, that can do more than “complete this text for me”?
The problem is you want to achieve a high level answer from a low level model, it doesn’t matter how much you change models if you keep to low parameter ones, you need to use big ones like the ones used in their data centers.
I’ve used 13B models with somewhat good results, I only tried once the mistral 8x7B and it was amazing the responses it gave.
But this was using llamacpp offloading some layers to the GPU and just the base model, no training.
Also, how did you connected the llm to your notes? Did you trained a lora? Used embeddings? Or were your notes just fed via the context?
IIRC the last two are basically the same and are limited to what your model accepts, usually 2048 tokens, which might be enough for a one chat with a not, but not enough for large amounts of notes.