Thoughts
Thinking about the Discord Ripgrep comment again.
At risk of deconstructing the humor,
It's absurd to me for 3 reasons.
1. rg and Elasticsearch do different things. They're not comparable. You would have to build a layer to move messages from the DB to files so that you could use rg, then build an API to expose the result of rg to the client.
2. rg is O(n). Discord has billions of messages. You could obviously chunk them by server and time and build an index but like. That's what Ellastisearch does.
3. The sheer scale. I just I can't. I You can't put like. Even if you did 1 and 2 and built this cursed index-adaption layer for storing messages in plaintext files and indexing them, like. You're at risk of like. You can't put that many files on a hard drive. You have to build a distributed replication layer.
Like, building 1, 2, and 3 from scratch is so hard that you might as well re-build ripgrep while you're at it.
=> https://thoughts.learnerpages.com/?show=acafe37f-063e-4276-8373-58a9d0d7a1af