Commit graph

624 commits

Author SHA1 Message Date
Ekaterina Vaartis cc3319ac1d Make chunk size configurable 2022-06-29 20:49:45 +01:00
Ekaterina Vaartis bac70a2bc1 Implement suggestions from the Meilisearch MR
- Index unlisted posts
- Move version check outside of the streaming and only do it once
- Use a PUT request instead of checking manually if there is need to insert
- Add error handling, sort of
2022-06-29 20:49:45 +01:00
Ekaterina Vaartis 0769f06bd1 Style fixes 2022-06-29 20:49:00 +01:00
Ekaterina Vaartis 86971fceaa Support reindexing meilisearch >=0.24.0
It has has a different error code key
2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 9e7d7ebd48 Add a reindex option
Signed-off-by: Ekaterina Vaartis <vaartis@kotobank.ch>
2022-06-29 20:48:44 +01:00
Ekaterina Vaartis 5ed1759091 Reorder ranking rules for (maybe) better results 2022-06-29 20:48:44 +01:00
Ekaterina Vaartis d1079f1aa3 Add the meilisearch.stats command 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis d5cc272a91 Add a message with a count of posts to index 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis dbf556cdcf Implement meilisearch auth 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 5360cc1097 Make indexing logs rewrite themselves 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 0cf3654907 Rework task indexing to share code with the main module
The code in the main module now scrubs new posts too
2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 117f525fd6 Adjust content indexing to skip more unneeded stuff 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 14ef6ce80f Mark only content as searchable for meilisearch 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 52a872432d Make the chunk size smaller 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis a586ce0ddd Use content instead of source and scrub it 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis c3a04166a0 Tweak search ordering to hopefully return newer results 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 7b3701e6b9 Make meilisearch sort on publish date converted to unix time 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis abf82a63ec Make the indexing batch differently and more, show number indexed 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis e5ac2ffa07 Ensure only indexing public posts and implement clearing and delete 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis 41db5c8653 Add logging to milisiearch index and make it use desc(id) 2022-06-29 20:48:29 +01:00
Ekaterina Vaartis e961cf2689 Initial meilisearch implementation, doesn't delete posts yet 2022-06-29 20:48:28 +01:00
FloatingGhost 12a171844b Merge remote-tracking branch 'upstream/develop' into develop 2021-12-30 18:05:22 +00:00
FloatingGhost 415d9dd6f6 Merge remote-tracking branch 'origin/feature/elasticsearch' into develop 2021-12-16 16:13:41 +00:00
Alex Gleason e1b89fe3aa
Merge remote-tracking branch 'origin/develop' into live-dashboard 2021-12-15 19:05:36 -05:00
Alex Gleason 29d80b39f2
Add Phoenix LiveDashboard
Co-authored-by: Egor Kislitsyn <egor@kislitsyn.com>
2021-12-15 19:05:27 -05:00
sadposter 8378c3a8bc extra cool 2021-12-14 14:48:24 +00:00
FloatingGhost 9d5d964bf8 merge 2021-12-14 14:28:15 +00:00
sadposter cc4c5f22f4 fix inbound federation 2021-12-14 14:27:42 +00:00
sadposter 8f78cb3050 Merge branch 'develop' of gitssh.ihatebeinga.live:IHBAGang/pleroma into develop 2021-12-14 14:01:09 +00:00
sadposter 5d901c337e fix inbound federation 2021-12-14 13:58:16 +00:00
FloatingGhost 703f53c08f Search through users and hashtags as well 2021-12-14 13:55:14 +00:00
FloatingGhost a85bf5929b add hashtag indexing 2021-12-14 13:55:14 +00:00
FloatingGhost 6ca70833f5 add user import 2021-12-14 13:55:14 +00:00
FloatingGhost c03e8d46e8 Search through users and hashtags as well 2021-12-14 13:53:46 +00:00
FloatingGhost 8547cefaff add hashtag indexing 2021-12-14 12:37:10 +00:00
FloatingGhost 6bb19454fd add user import 2021-12-13 20:27:35 +00:00
sadposter e835212a7d enforce visibility 2021-12-12 20:12:06 +00:00
sadposter 23c758bbe7 enforce visibility 2021-12-12 18:25:20 +00:00
FloatingGhost de60b1f3c1 make search provider configurable 2021-12-12 17:35:35 +00:00
FloatingGhost 9002e5155b Add import functionality 2021-12-12 17:35:35 +00:00
FloatingGhost ed3a866f94 make search provider configurable 2021-12-12 17:23:44 +00:00
FloatingGhost c684265585 Add import functionality 2021-12-11 17:36:49 +00:00
sadposter 6a6260f64e re-enable deactivated check 2021-12-06 12:03:16 +00:00
sadposter ca2bd8b72b remove deactivated_users call 2021-12-06 10:18:16 +00:00
sadposter bb54720778 add debug 2021-12-05 21:56:11 +00:00
sadposter eb1c7d75f4 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-10-16 14:08:04 +01:00
Haelwenn (lanodan) Monnier a17910a6c6
CI: Bump lint stage to elixir-1.12
Elixir 1.12 changed formatting rules, this allows to avoid having to rollback to run `mix format`
2021-10-06 08:11:05 +02:00
sadposter b202cf15d6 Merge branch 'develop' of https://git.pleroma.social/pleroma/pleroma into develop 2021-08-30 22:04:42 +01:00
someone cc4f20b130
mix pleroma.database set_text_search_config now runs concurrently and infinitely
Backport of: https://git.pleroma.social/pleroma/pleroma/-/merge_requests/3514
2021-08-28 18:29:43 +02:00
someone 4b940e441a mix pleroma.database set_text_search_config now runs concurrently and infinitely 2021-08-15 13:49:12 -04:00