Commit graph

1521 commits

Author SHA1 Message Date
ilja 7695010268 Prune Objects --keep-threads option (#350)
This adds an option to the prune_objects mix task.
The original way deleted all non-local public posts older than a certain time frame.
Here we add a different query which you can call using the option --keep-threads.

We query from the activities table all context id's where
    1. the newest activity with this context is still old
    2. none of the activities with this context is is local
    3. none of the activities with this context is bookmarked
and delete all objects with these contexts.

The idea is that posts with local activities (posts, replies, likes, repeats...) may be interesting to keep.
Besides that, a post lives in a certain context (the thread), so we keep the whole thread as well.

Caveats:
* ~~Quotes have a different context. Therefore, when someone quotes a post, it's possible the quoted post will still be deleted.~~ fixed in https://akkoma.dev/AkkomaGang/akkoma/pulls/379
* Although undocumented (in docs/docs/administration/CLI_tasks/database.md/#prune-old-remote-posts-from-the-database), the 'normal' delete action still kept old remote non-public posts. I added an option to keep this behaviour, but this also means that you now have to explicitly provide that option. **This could be considered a breaking change!**
* ~~Note that this removes from the objects table, but not from the activities.~~ See https://akkoma.dev/AkkomaGang/akkoma/pulls/427 for that.

Some statistics from explain analyse:
(cost=1402845.92..1933782.00 rows=3810907 width=62) (actual time=2562455.486..2562455.495 rows=0 loops=1)
 Planning Time: 505.327 ms
 Trigger for constraint chat_message_references_object_id_fkey: time=651939.797 calls=921740
 Trigger for constraint deliveries_object_id_fkey: time=52036.009 calls=921740
 Trigger for constraint hashtags_objects_object_id_fkey: time=20665.778 calls=921740
 Execution Time: 3287933.902 ms

***
**TODO**
1. [x] **Question:** Is it OK to keep it like this in regard to quote posts? If not (ie post quoted by local users should also be kept), should we give quotes the same context as the post they are quoting? (If we don't want to give them the same context, I'll have to see how/if I can do it without being too costly)
    * See https://akkoma.dev/AkkomaGang/akkoma/pulls/379
2. [x] **Question:** the "original" query only deletes public posts (this is undocumented, but you can check the code). This new one doesn't care for scope. From the docs I get that the idea is that posts can be refetched when needed. But I have from a trusted source that Pleroma can't refetch non-public posts. I assume that's the reason why they are kept here. I see different options to deal with this
    1. ~~We keep it as currently implemented and just don't care about scope with this option~~
    2. ~~We add logic to not delete non-public posts either (I'll have to see how costly that becomes)~~
    3. We add an extra --keep-non-public parameter. This is technically speaking breakage (you didn't have to provide a param before for this, now you do), but I'm inclined to not care much because it wasn't documented nor tested in the first place.
3. [x] See if we can do the query using Elixir
4. [x] Test on a bigger DB to see that we don't run into a timeout
5. [x] Add docs

Co-authored-by: ilja <git@ilja.space>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/350
Co-authored-by: ilja <akkoma.dev@ilja.space>
Co-committed-by: ilja <akkoma.dev@ilja.space>
2023-01-09 22:15:41 +00:00
FloatingGhost 6a333ade7f Fix task name for robotstxt
Fixes #408
2023-01-01 18:54:08 +00:00
timorl bca1c43dcb Add docs about emoji stealing (#364)
I managed to steal some emoji, but I had to figure out the specifics the hard way. This should make it easier for future criminals.

Feel free to close if this documentation was omitted on purpose, I can imagine some reasons for why it might have.

Co-authored-by: timorl <timorl@disroot.org>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/364
Co-authored-by: timorl <timorl+akkomadev@disroot.org>
Co-committed-by: timorl <timorl+akkomadev@disroot.org>
2022-12-30 02:58:06 +00:00
floatingghost bdc676e433 Merge pull request 'docs/installation: update comment to reflect flavour change' (#394) from norm/akkoma:flavour-fix into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/394
2022-12-30 02:54:09 +00:00
floatingghost 5624366056 Merge pull request 'docs: fedora install errata' (#398) from acuteaura/akkoma:acuteaura-patch-1 into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/398
2022-12-30 02:50:34 +00:00
FloatingGhost 1121deb078 Document instance reboots 2022-12-29 20:24:04 +00:00
FloatingGhost 5a405bdadf document dump_to_file and load_from_file 2022-12-29 20:00:04 +00:00
FloatingGhost e66bcb64a4 Check out the latest tag on update 2022-12-29 15:42:25 +00:00
Stefan 11ec4e1b8f clean-up docs to avoid mismatches in BE and FE. Clearly state that stable-versions are installed 2022-12-29 15:41:20 +00:00
acuteaura e392662d76 docs: fedora install errata 2022-12-25 15:32:57 +00:00
Francis Dinh 6610a1d5fb
docs/installation: update comment to reflect flavour change
The comment still says the flavour is `amd64-musl` when it was updated
to just `amd64` in 64ccdadad3.
2022-12-22 19:18:54 -05:00
YokaiRick 64ccdadad3 switch default flavor to amd64
Most ppl probably need to set it to amd64.
This would help to remove some confusion while installing akkoma
2022-12-22 21:25:31 +00:00
ilja c092fc9fd6 Add translation module for Argos Translate (#351)
Argos Translate is a Python module for translation and can be used as a command line tool.

This is also the engine for LibreTranslate, for which we already have a module.
Here we can use the engine directly from our server without doing requests to a third party or having to install our own LibreTranslate webservice (obviously you do have to install Argos Translate).

One thing that's currently still missing from Argos Translate is auto-detection of languages (see <https://github.com/argosopentech/argos-translate/issues/9>). For now, when no source language is provided, we just return the text unchanged, supposedly translated from the target language. That way you get a near immediate response in pleroma-fe when clicking Translate, after which you can select the source language from a dropdown.

Argos Translate also doesn't seem to handle html very well. Therefore we give admins the option to strip the html before translating. I made this an option because I'm unsure if/how this will change in the future.

Co-authored-by: ilja <git@ilja.space>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/351
Co-authored-by: ilja <akkoma.dev@ilja.space>
Co-committed-by: ilja <akkoma.dev@ilja.space>
2022-12-19 13:06:39 +00:00
Paul Dawson eb9ef59d50 Remove legacy references to FE that is not officially supported 2022-12-16 08:08:00 -06:00
FloatingGhost 584f99b69d fix markdown link 2022-12-16 13:24:18 +00:00
FloatingGhost 48d302a60f allow disabling prometheus entirely 2022-12-16 11:17:04 +00:00
FloatingGhost d1a0d93bf7 document prometheus 2022-12-16 10:24:36 +00:00
Norm 53f21489a2 Update the "Updating your instance" docs (#361)
This makes them consistent with the update instructions that are in the
release announcements.

Also adds in the command to update the frontend as well.

Co-authored-by: Francis Dinh <normandy@biribiri.dev>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/361
Co-authored-by: Norm <normandy@biribiri.dev>
Co-committed-by: Norm <normandy@biribiri.dev>
2022-12-14 03:03:08 +00:00
FloatingGhost 9c71782861 Test removed HTTP adapter 2022-12-11 23:50:31 +00:00
FloatingGhost f752126427 Remove quack, ensure adapter is finch 2022-12-11 23:22:35 +00:00
FloatingGhost affc910372 Remove hackney/gun in favour of finch 2022-12-11 19:19:31 +00:00
r3g_5z fbfffccc1d
Add dark and light theme mode to docs, detection, and button
my eyes hurt

Signed-off-by: r3g_5z <june@girlboss.ceo>
2022-12-09 22:51:43 -05:00
FloatingGhost 9db4c2429f Remove FollowBotPolicy 2022-12-09 19:59:27 +00:00
FloatingGhost 221a95b860 Document custom.css 2022-12-07 11:45:53 +00:00
floatingghost 09326ffa56 Diagnostics tasks (#348)
a bunch of ways to get query plans to help with debugging

Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/348
2022-12-07 11:12:34 +00:00
floatingghost 4f2f2c9125 Merge pull request 'Small improvements to the Gentoo installation isntructions' (#335) from timorl/akkoma:i-use-gentoo-btw into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/335
2022-12-07 11:07:06 +00:00
ilja fdf33392b3 DOCS: backup restore improvements (#332)
Mostly add how to speed up restoration by adding activities_visibility_index later. Also some small other improvements.

This is based on what I did on a Pleroma instance. I assume the activities_visibility_index taking so long is still true for Akkoma, but can't really test because I don't have a big enough Akkoma DB yet 🙃

Co-authored-by: ilja <git@ilja.space>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/332
Reviewed-by: floatingghost <hannah@coffee-and-dreams.uk>
Co-authored-by: ilja <akkoma.dev@ilja.space>
Co-committed-by: ilja <akkoma.dev@ilja.space>
2022-12-07 11:05:35 +00:00
FloatingGhost b70a60c6c5 Doc branding 2022-12-05 13:45:36 +00:00
floatingghost c62e1e3ad5 varnish config/docs (#342)
Co-authored-by: Mark Felder <feld@feld.me>
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/342
2022-12-05 13:39:27 +00:00
timorl d923cb96b1
Small improvements to the Gentoo installation isntructions 2022-12-04 16:37:49 +01:00
floatingghost 6b882a2c0b Purge Rejected Follow requests in daily task (#334)
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/334
2022-12-03 23:17:43 +00:00
floatingghost db60640c5b Fixing up deletes a bit (#327)
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/327
2022-12-01 15:00:53 +00:00
Norm b188ac3f21 docs: Remove quarantine section
Quarantining was deprecated back in 2022.08.

Also added that SimplePolicy's `reject` also prevents outbound federation to servers listed there.
2022-11-29 18:31:24 +00:00
floatingghost 98a21debf9 normalise markup by default (#316)
why was this _not_ default?

honestly i'm surprised pleroma hasn't exploded yet

Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/316
2022-11-26 21:06:20 +00:00
@r3g_5z@plem.sapphic.site 565ead8397 minor-changes (#313)
Only real change here is making MRF rejects log as debug instead of info (https://akkoma.dev/AkkomaGang/akkoma/issues/234)

I don't know if it's the best way to do it, but it seems it's just MRF using this and almost always this is intended.

The rest are just minor docs changes and syncing the restricted nicknames stuff.

I compiled and ran my changes with Docker and they all work.

Co-authored-by: r3g_5z <june@terezi.dev>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/313
Co-authored-by: @r3g_5z@plem.sapphic.site <june@girlboss.ceo>
Co-committed-by: @r3g_5z@plem.sapphic.site <june@girlboss.ceo>
2022-11-26 19:27:58 +00:00
FloatingGhost ec1d903f2e Note that openbsd needs erlang-wx 2022-11-25 15:24:39 +00:00
floatingghost 2fe1484ed3 http timeout config (#307)
Ref https://meta.akkoma.dev/t/increase-timeout-on-libretranslate-request-how/156/2

Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/307
2022-11-24 12:27:16 +00:00
FloatingGhost cc75b313f3 Add favicon, frontend docs 2022-11-20 21:44:32 +00:00
@r3g_5z@plem.sapphic.site 0e4c201f8d HTTP header improvements (#294)
- Drop Expect-CT

Expect-CT has been redundant since 2018 when Certificate Transparency became mandated and required for all CAs and browsers. This header is only implemented in Chrome and is now deprecated. HTTP header analysers do not check this anymore as this is enforced by default. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expect-CT

- Raise HSTS to 2 years and explicitly preload

The longer age for HSTS, the better. Header analysers prefer 2 years over 1 year now as free TLS is very common using Let's Encrypt.
For HSTS to be fully effective, you need to submit your root domain (domain.tld) to https://hstspreload.org. However, a requirement for this is the "preload" directive in Strict-Transport-Security. If you do not have "preload", it will reject your domain.

- Drop X-Download-Options

This is an IE8-era header when Adobe products used to use the IE engine for making outbound web requests to embed webpages in things like Adobe Acrobat (PDFs). Modern apps are using Microsoft Edge WebView2 or Chromium Embedded Framework. No modern browser checks or header analyser check for this.

- Set base-uri to 'none'

This is to specify the domain for relative links (`<base>` HTML tag). pleroma-fe does not use this and it's an incredibly niche tag.

I use all of these myself on my instance by rewriting the headers with zero problems. No breakage observed.

I have not compiled my Elixr changes, but I don't see why they'd break.

Co-authored-by: r3g_5z <june@terezi.dev>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/294
Co-authored-by: @r3g_5z@plem.sapphic.site <june@terezi.dev>
Co-committed-by: @r3g_5z@plem.sapphic.site <june@terezi.dev>
2022-11-20 21:20:06 +00:00
r3g_5z f90552f62e
Drop XSS auditor
It's deprecated, removed in some, by all modern browsers and is known
to create XSS vulnerabilities in itself.

Signed-off-by: r3g_5z <june@terezi.dev>
2022-11-19 20:40:20 -05:00
FloatingGhost fb5f846e8c Add languages to cheatsheet 2022-11-18 11:22:30 +00:00
Norm 14c1a4220b docs: Update list of clients (#284)
In addition to making the page refer to Akkoma instead of Pleroma, I've
also removed clients that were not updated in a year or more and updated
links to websites and the contact links of authors.

Also removed language that suggested these clients are in any way
"officially supported".

Co-authored-by: Francis Dinh <normandy@biribiri.dev>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/284
Co-authored-by: Norm <normandy@biribiri.dev>
Co-committed-by: Norm <normandy@biribiri.dev>
2022-11-18 11:19:37 +00:00
floatingghost 9deae8c533 Merge pull request 'docs: Update links to list of akkoma instances' (#278) from norm/akkoma:update-akkoma-list-urls into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/278
2022-11-16 10:16:27 +00:00
FloatingGhost d4ca1217d3 Be very specific about the double-quotes in strings 2022-11-16 10:13:41 +00:00
Francis Dinh 7a833aff90
docs: Update links to list of akkoma instances
The old links were for Pleroma instances and one of them isn't even active anymore.
2022-11-15 07:51:19 -05:00
FloatingGhost 893bfde66f Remove references to soykaf
Fixes #271
2022-11-14 00:01:31 +00:00
FloatingGhost 634463ff64 fix requirements 2022-11-11 16:07:07 +00:00
FloatingGhost dcc36df8cf add manual deploy for docs 2022-11-10 10:55:57 +00:00
FloatingGhost 539c6d6666 update requirements.txt 2022-11-10 03:40:36 +00:00
FloatingGhost 66eb844bd2 Update documentation builder 2022-11-10 03:38:10 +00:00