Does somebody just need to buy a lot of hard drives and data tapes, and program a bunch of raspberry pi to download everything it can find?
Edit: What I’m specifically asking about is the feature reddit had to search the site itself. Obviously for reddit this process is much simpler, since they’re just searching their own database.
This seems to work pretty well
There’s one already:
Sadly that just searches instances and not posts/comments?
This doesn’t seems to work very well. I searched for Diablo and got 0 results
Is a search engine what is needed? I have some thoughts on how to make an effective one but it won’t be cheap. Essentially drunk from the activity pub firehouse and index everything
idk, ddg already works pretty well…
I haven’t heard of ddg yet, do you have a link or somewhere you can point me to?
But otherwise I’m actually trying to look at creating a self-hosted search engine that anyone can run along side lemmy or any other fediverse instance. Idea is to have it work similar to Google but just for fediverse. Just getting my build environment setup and hope to start development in the next couple of days as I research ActivityPub and find out how it works in more detail.
Lol DuckDuckGo. Drr… was thinking there was a dedicated service already for just the fediverse.
Yeah, my point is that ddg already searches the fediverse. To limit the searches just to the fediverse, I’m sure there’s some OSINT google-fu you could use. Or email them a feature request asking for a way to limit it to fediverse sites only. Or fork them.
Ya I’m hoping for something closer to what I do today with Reddit, err used to do ;) Where I can just type in “best VPN providers reddit” and see a listing of posts / and a short blurb on the search results page itself. And ya trying to filter out all of the blog spam from the rest of the web. Since there’s no one single site for the fediverse it becomes a little trickier, especially for the layman to find the content they want.
I can just type in "best VPN providers reddit”
lol that’ll just give you a bunch of spam and ads for shitty VPNs like Nord
True but not everyone has the google-fu to filter their search results to weed out spam, etc… So how can we make things easier for the average Joe to be able to find the reviews/feedback/advice that they’re searching for without someone raising the same questions here everyday?
I.e. what if there was a search engine specifically built to just search all of the comments and posts in the fediverse? What if that engine could be hosted in your own home so you don’t have to worry if someone is snooping on your search history? What if there were no ads of any kind in those same results?
