Compare commits

...

54 Commits

Author SHA1 Message Date
Sam Therapy 8d299302b6
Merge branch 'develop' of https://akkoma.dev/AkkomaGang/akkoma into froth-akkoma 2023-02-18 15:11:56 +01:00
FloatingGhost ef279ac53f build musl on develop 2023-02-15 22:09:31 +00:00
FloatingGhost b4e37b03d8 use a slightly earlier alpine version 2023-02-15 22:08:55 +00:00
FloatingGhost f92484fd01 add version in changelog 2023-02-11 11:19:33 +00:00
FloatingGhost 6e07ed6ea9 fix docker tag 2023-02-11 10:58:53 +00:00
FloatingGhost d2b57a7f9e bump elixir version in CI 2023-02-11 10:56:53 +00:00
FloatingGhost 439e915531 bump version 2023-02-11 10:48:52 +00:00
ilja b71db2f82d create_service_actor is now type Application
This is used for internal fetch and for relay. Both represent the instance and therefore are an aplication.
2023-02-04 21:00:21 +00:00
floatingghost aeb68a0ad1 paginate follow requests (#460)
matches https://docs.joinmastodon.org/methods/follow_requests/#get mostly

Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/460
2023-02-04 20:51:17 +00:00
floatingghost 56c37dc6b3 Merge pull request 'DOCS: Add info on fe to setup of dev env' (#456) from ilja/akkoma:docs_add_fe_to_akkoma_dev into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/456
2023-01-30 11:28:28 +00:00
floatingghost 3405623d46 Merge pull request 'Docs: Add more info to the development index file' (#451) from ilja/akkoma:docs_add_some_developer_docs into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/451
2023-01-30 11:27:23 +00:00
floatingghost 702979bca3 Merge pull request 'Docs: Changes to pgtune docs' (#449) from ilja/akkoma:docs_small_adition_to_pgtune into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/449
2023-01-30 11:25:46 +00:00
ilja 7e3ede02f7 Add info on fe to setup of dev env
I added info about installing front ends from the development branch

I also rearanged the list of exceptions (what's different than "normal" installation)
so the order is closer to how you'd encounter things in the installation docs + small fixes
2023-01-29 08:29:06 +01:00
floatingghost d601ddeb91 Merge pull request 'Make default outgoing-blocks setting off' (#454) from Seirdy/akkoma:outgoing-blocks-default-off into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/454
2023-01-27 10:06:49 +00:00
Seirdy 676cc0d0d7
Make default outgoing-blocks setting off
This should help mitigate negative impacts related to block-retaliation
and block-circumvention when blocks become visible to the blocked party.
Instances interested in broadcasting blocks can turn this on if they
wish. This should have always been the default.

See also: https://akkoma.dev/AkkomaGang/akkoma-fe/pulls/274
2023-01-26 22:01:22 -08:00
ilja ce6f652a9a Add more info to the development index file
Credit where credit is due; I inspired myself by looking at the yunohost docs
* https://yunohost.org/en/dev
* https://yunohost.org/en/packaging_apps_start

I try to be inviting to new developers and guide them in their first steps into Akkoma development.
I try to keep the page itself as short as possible and link to relevant places.
That way people can quickly skim over parts that they don't need, while people who do need more can simply follow the links.
2023-01-26 13:40:51 +01:00
ilja 377527ea03 Changes to pgtune docs
I experienced that it may be better to tell pgtune you have lower resoures than what you have when you have other services running.
I added that now.

I also moved the examples as part of the pgtune section.
2023-01-26 10:11:29 +01:00
FloatingGhost 153539a246 add changelog entry 2023-01-15 19:02:58 +00:00
FloatingGhost d394ab0a8a Merge branch 'develop' of akkoma.dev:AkkomaGang/akkoma into develop 2023-01-15 18:58:26 +00:00
FloatingGhost 90088cce11 Support TLD wildcards in MRF matches
Fixes #431
2023-01-15 18:57:49 +00:00
floatingghost 63ce25f32c Merge pull request 'Correct og:description tag in static-fe' (#373) from sfr/akkoma:fix/og-description into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/373
2023-01-15 18:15:20 +00:00
sfr 20cd8a0fc4 URL encode remote emoji pack names (#362)
fix #246

Co-authored-by: Sol Fisher Romanoff <sol@solfisher.com>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/362
Co-authored-by: sfr <sol@solfisher.com>
Co-committed-by: sfr <sol@solfisher.com>
2023-01-15 18:14:04 +00:00
floatingghost 0fb2042f2c Merge pull request 'Remove refences to ARM OTP builds in install guide' (#432) from norm/akkoma:remove-arm-otp into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/432
2023-01-15 18:11:53 +00:00
floatingghost 0c8da6466e Merge pull request 'Update OpenBSD docs' (#414) from lechindianer/akkoma:update-openbsd-docs into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/414
2023-01-15 18:09:32 +00:00
floatingghost 975bc6d7e8 Merge pull request 'fix: Give error message to users when address has already been validated' (#435) from cheerfulstoic/akkoma:develop into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/435
2023-01-15 18:06:12 +00:00
FloatingGhost 2fc5fb7f5a Add changelog entry 2023-01-15 18:05:02 +00:00
FloatingGhost f3c118ca23 Mix format 2023-01-15 18:00:03 +00:00
FloatingGhost 0d342a35e3 add contentMap to schema 2023-01-15 17:59:19 +00:00
Brian Underwood 7ca9ce9d67 fix: Give error message to users when address has already been validated
Plus other errors.
2023-01-12 22:08:10 +01:00
FloatingGhost 65e8e8fb6d Merge branch 'language-on-posts' into develop 2023-01-11 15:44:15 +00:00
FloatingGhost ff5793198f add inbound language test 2023-01-11 15:42:13 +00:00
FloatingGhost 78c44f31ca fix no-language-specified federation 2023-01-11 15:25:34 +00:00
floatingghost 260c87006e revert ae54c06bb4
revert Language code on mastoapi statuses (#433)

Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/433
2023-01-10 15:00:47 +00:00
floatingghost ae54c06bb4 Language code on mastoapi statuses (#433)
Co-authored-by: FloatingGhost <hannah@coffee-and-dreams.uk>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/433
2023-01-10 13:45:43 +00:00
FloatingGhost 22068f0853 fix unused variable warnings 2023-01-10 10:58:17 +00:00
FloatingGhost 7eebaa7a18 bump mock 2023-01-10 10:48:14 +00:00
FloatingGhost cc63a89b5d Fix tests 2023-01-10 10:29:17 +00:00
FloatingGhost f86bf16430 Add language support on /api/v1/statuses 2023-01-10 10:29:17 +00:00
floatingghost 6965a2f163 Merge pull request 'use postgres 15 in CI' (#411) from ci-postgres-15 into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/411
2023-01-09 22:16:43 +00:00
ilja 7695010268 Prune Objects --keep-threads option (#350)
This adds an option to the prune_objects mix task.
The original way deleted all non-local public posts older than a certain time frame.
Here we add a different query which you can call using the option --keep-threads.

We query from the activities table all context id's where
    1. the newest activity with this context is still old
    2. none of the activities with this context is is local
    3. none of the activities with this context is bookmarked
and delete all objects with these contexts.

The idea is that posts with local activities (posts, replies, likes, repeats...) may be interesting to keep.
Besides that, a post lives in a certain context (the thread), so we keep the whole thread as well.

Caveats:
* ~~Quotes have a different context. Therefore, when someone quotes a post, it's possible the quoted post will still be deleted.~~ fixed in https://akkoma.dev/AkkomaGang/akkoma/pulls/379
* Although undocumented (in docs/docs/administration/CLI_tasks/database.md/#prune-old-remote-posts-from-the-database), the 'normal' delete action still kept old remote non-public posts. I added an option to keep this behaviour, but this also means that you now have to explicitly provide that option. **This could be considered a breaking change!**
* ~~Note that this removes from the objects table, but not from the activities.~~ See https://akkoma.dev/AkkomaGang/akkoma/pulls/427 for that.

Some statistics from explain analyse:
(cost=1402845.92..1933782.00 rows=3810907 width=62) (actual time=2562455.486..2562455.495 rows=0 loops=1)
 Planning Time: 505.327 ms
 Trigger for constraint chat_message_references_object_id_fkey: time=651939.797 calls=921740
 Trigger for constraint deliveries_object_id_fkey: time=52036.009 calls=921740
 Trigger for constraint hashtags_objects_object_id_fkey: time=20665.778 calls=921740
 Execution Time: 3287933.902 ms

***
**TODO**
1. [x] **Question:** Is it OK to keep it like this in regard to quote posts? If not (ie post quoted by local users should also be kept), should we give quotes the same context as the post they are quoting? (If we don't want to give them the same context, I'll have to see how/if I can do it without being too costly)
    * See https://akkoma.dev/AkkomaGang/akkoma/pulls/379
2. [x] **Question:** the "original" query only deletes public posts (this is undocumented, but you can check the code). This new one doesn't care for scope. From the docs I get that the idea is that posts can be refetched when needed. But I have from a trusted source that Pleroma can't refetch non-public posts. I assume that's the reason why they are kept here. I see different options to deal with this
    1. ~~We keep it as currently implemented and just don't care about scope with this option~~
    2. ~~We add logic to not delete non-public posts either (I'll have to see how costly that becomes)~~
    3. We add an extra --keep-non-public parameter. This is technically speaking breakage (you didn't have to provide a param before for this, now you do), but I'm inclined to not care much because it wasn't documented nor tested in the first place.
3. [x] See if we can do the query using Elixir
4. [x] Test on a bigger DB to see that we don't run into a timeout
5. [x] Add docs

Co-authored-by: ilja <git@ilja.space>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/350
Co-authored-by: ilja <akkoma.dev@ilja.space>
Co-committed-by: ilja <akkoma.dev@ilja.space>
2023-01-09 22:15:41 +00:00
floatingghost 357f80a714 Merge pull request 'Changed references of "Pleroma" to "Akkoma" in email text' (#428) from knova/akkoma:develop into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/428
2023-01-09 22:13:45 +00:00
darkkirb a8cd859ef9 Use actual ISO8601 timestamps for masto API (#425)
Some users post posts with spoofed timestamp, and some clients will have issues with certain dates. Tusky for example crashes if the date is any sooner than 1 BCE (“year zero” in the representation).

I limited the range of what is considered a valid date to be somewhere between the years 1583 and 9999 (inclusive).

The numbers have been chosen because:

- ISO 8601 only allows years before 1583 with “mutual agreement”
- Years after 9999 could cause issues with certain clients as well

Co-authored-by: Charlotte 🦝 Delenk <lotte@chir.rs>
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/425
Co-authored-by: darkkirb <lotte@chir.rs>
Co-committed-by: darkkirb <lotte@chir.rs>
2023-01-09 22:12:28 +00:00
floatingghost 0d56adc16b Merge pull request 'exiftool doesn’t support JPEG XL either' (#426) from darkkirb/akkoma:exiftools-no-jxl into develop
Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/426
2023-01-09 22:10:54 +00:00
Norm eb55472450 Remove refences to ARM OTP builds in install guide
Akkoma currently doesn't provide ARM OTP builds unlike Pleroma, so it's best to update the install guide to reflect that.

Ref: https://akkoma.dev/AkkomaGang/akkoma/issues/424
2023-01-09 20:11:33 +00:00
knova 13d943667e Changed references of "Pleroma" to "Akkoma" in email text
I know this is probably small peanuts in the grand scheme of things, but it bugged me when I was messing around with my own Akkoma instance.
2023-01-08 03:29:09 +00:00
Charlotte 🦝 Delenk f2b925f32c
exiftool doesn’t support JPEG XL either 2023-01-07 14:49:58 +01:00
ihor b98fe4476c fix "exiftool not support svg files' (#421)
Faced with this issue today, Pleroma responds with status 400 (Bad request) if Exiftool.StripLocation is added to the list of filter modules for uploads. Here is logs:

```
13:27:25.201 [info] POST /api/v1/media

13:27:25.232 request_id=FzdspaAnrA6cyv0APgVR [error] Elixir.Pleroma.Upload.Filter: Filter Elixir.Pleroma.Upload.Filter.Exiftool.StripLocation failed: {:error, "Elixir.Pleroma.Upload.Filter.Exiftool.StripLocation: %ErlangError{original: :enoent}"}

13:27:25.232 request_id=FzdspaAnrA6cyv0APgVR [error] Elixir.Pleroma.Upload store (using Pleroma.Uploaders.Local) failed: "Elixir.Pleroma.Upload.Filter.Exiftool.StripLocation: %ErlangError{original: :enoent}"
```

# This fix solves this problem.

Reviewed-on: https://akkoma.dev/AkkomaGang/akkoma/pulls/421
Co-authored-by: ihor <ikandreew@gmail.com>
Co-committed-by: ihor <ikandreew@gmail.com>
2023-01-05 15:22:48 +00:00
Pascal Schmid eb1b9c4155 Fix PostgreSQL docs
Without creating the directory manually and setting it as user home folder
the commands afterwards won't be executable.
2023-01-02 13:12:54 +01:00
Pascal Schmid fcce355112 Delete broken config example
httpd won't start with the empty types block.
2023-01-02 13:12:20 +01:00
Pascal Schmid ef1c68a8e9 Format docs 2023-01-02 13:12:17 +01:00
Pascal Schmid d427c23e56 Use current package config 2023-01-02 12:56:49 +01:00
Pascal Schmid 769b5969a8 Update referenced OpenBSD version 2023-01-02 12:50:30 +01:00
FloatingGhost e03206a9a0 use postgres 15 in CI 2022-12-31 18:29:16 +00:00
Sol Fisher Romanoff 1d884fd914
Correct og:description tag in static-fe 2022-12-30 07:14:54 +02:00
49 changed files with 914 additions and 145 deletions

View File

@ -41,7 +41,7 @@ variables:
services:
postgres:
image: postgres:13
image: postgres:15
when:
event:
- pull_request
@ -95,7 +95,7 @@ pipeline:
# Canonical amd64
ubuntu22:
image: hexpm/elixir:1.14.2-erlang-25.1.2-ubuntu-jammy-20220428
image: hexpm/elixir:1.14.3-erlang-25.2.2-ubuntu-jammy-20221130
<<: *on-release
environment:
MIX_ENV: prod
@ -122,7 +122,7 @@ pipeline:
- /bin/sh /entrypoint.sh
debian-bullseye:
image: hexpm/elixir:1.14.2-erlang-25.1.2-debian-bullseye-20221004
image: hexpm/elixir:1.14.3-erlang-25.2.2-debian-bullseye-20230109
<<: *on-release
environment:
MIX_ENV: prod
@ -151,8 +151,8 @@ pipeline:
# Canonical amd64-musl
musl:
image: hexpm/elixir:1.14.2-erlang-25.1.2-alpine-3.16.2
<<: *on-stable
image: hexpm/elixir:1.14.3-erlang-25.2.2-alpine-3.15.6
<<: *on-release
environment:
MIX_ENV: prod
commands:
@ -167,7 +167,7 @@ pipeline:
release-musl:
image: akkoma/releaser
<<: *on-stable
<<: *on-release
secrets: *scw-secrets
commands:
- export SOURCE=akkoma-amd64-musl.zip

View File

@ -4,7 +4,7 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## Unreleased
## 2023.02
### Added
- Prometheus metrics exporting from `/api/v1/akkoma/metrics`
@ -14,6 +14,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Ability to "verify" links in profile fields via rel=me
- Mix tasks to dump/load config to/from json for bulk editing
- Followed hashtag list at /api/v1/followed\_tags, API parity with mastodon
- Ability to set posting language in the post form, API parity with mastodon
- Ability to match domains in MRF by a trailing wildcard
- Currently supported formats:
- `example.com` (implicitly matches `*.example.com`)
- `*.example.com`
- `example.*` (implicitly matches `*.example.*`)
### Removed
- Non-finch HTTP adapters
@ -27,9 +33,12 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
- Admin scopes will be dropped on create
- Rich media will now backoff for 20 minutes after a failure
- Quote posts are now considered as part of the same thread as the post they are quoting
- Extend the mix task `prune_objects` with options to keep more relevant posts
- Simplified HTTP signature processing
- Rich media will now hard-exit after 5 seconds, to prevent timeline hangs
- HTTP Content Security Policy is now far more strict to prevent any potential XSS/CSS leakages
- Follow requests are now paginated, matches mastodon API spec, so use the Link header to paginate.
- `internal.fetch` and `relay` actors are now represented with the actor type `Application`
### Fixed
- /api/v1/accounts/lookup will now respect restrict\_unauthenticated

View File

@ -359,7 +359,7 @@
config :pleroma, :activitypub,
unfollow_blocked: true,
outgoing_blocks: true,
outgoing_blocks: false,
blockers_visible: true,
follow_handshake_timeout: 500,
note_replies_output_limit: 5,

View File

@ -27,7 +27,7 @@ Replaces embedded objects with references to them in the `objects` table. Only n
## Prune old remote posts from the database
This will prune remote posts older than 90 days (configurable with [`config :pleroma, :instance, remote_post_retention_days`](../../configuration/cheatsheet.md#instance)) from the database, they will be refetched from source when accessed.
This will prune remote posts older than 90 days (configurable with [`config :pleroma, :instance, remote_post_retention_days`](../../configuration/cheatsheet.md#instance)) from the database. Pruned posts may be refetched in some cases.
!!! danger
The disk space will only be reclaimed after `VACUUM FULL`. You may run out of disk space during the execution of the task or vacuuming if you don't have about 1/3rds of the database size free.
@ -45,6 +45,9 @@ This will prune remote posts older than 90 days (configurable with [`config :ple
```
### Options
- `--keep-threads` - don't prune posts when they are part of a thread where at least one post has seen local interaction (e.g. one of the posts is a local post, or is favourited by a local user, or has been repeated by a local user...)
- `--keep-non-public` - keep non-public posts like DM's and followers-only, even if they are remote
- `--vacuum` - run `VACUUM FULL` after the objects are pruned
## Create a conversation for all existing DMs
@ -178,4 +181,4 @@ to the current day.
```sh
mix pleroma.database prune_task
```
```

View File

@ -6,6 +6,31 @@ Akkoma performance is largely dependent on performance of the underlying databas
[PgTune](https://pgtune.leopard.in.ua) can be used to get recommended settings. Be sure to set "Number of Connections" to 20, otherwise it might produce settings hurtful to database performance. It is also recommended to not use "Network Storage" option.
If your server runs other services, you may want to take that into account. E.g. if you have 4G ram, but 1G of it is already used for other services, it may be better to tell PGTune you only have 3G. In the end, PGTune only provides recomended settings, you can always try to finetune further.
### Example configurations
Here are some configuration suggestions for PostgreSQL 10+.
#### 1GB RAM, 1 CPU
```
shared_buffers = 256MB
effective_cache_size = 768MB
maintenance_work_mem = 64MB
work_mem = 13107kB
```
#### 2GB RAM, 2 CPU
```
shared_buffers = 512MB
effective_cache_size = 1536MB
maintenance_work_mem = 128MB
work_mem = 26214kB
max_worker_processes = 2
max_parallel_workers_per_gather = 1
max_parallel_workers = 2
```
## Disable generic query plans
When PostgreSQL receives a query, it decides on a strategy for searching the requested data, this is called a query plan. The query planner has two modes: generic and custom. Generic makes a plan for all queries of the same shape, ignoring the parameters, which is then cached and reused. Custom, on the contrary, generates a unique query plan based on query parameters.
@ -23,26 +48,3 @@ config :pleroma, Pleroma.Repo,
```
A more detailed explaination of the issue can be found at <https://blog.soykaf.com/post/postgresql-elixir-troubles/>.
## Example configurations
Here are some configuration suggestions for PostgreSQL 10+.
### 1GB RAM, 1 CPU
```
shared_buffers = 256MB
effective_cache_size = 768MB
maintenance_work_mem = 64MB
work_mem = 13107kB
```
### 2GB RAM, 2 CPU
```
shared_buffers = 512MB
effective_cache_size = 1536MB
maintenance_work_mem = 128MB
work_mem = 26214kB
max_worker_processes = 2
max_parallel_workers_per_gather = 1
max_parallel_workers = 2
```

View File

@ -1 +1,48 @@
This section contains notes and guidelines for developers.
# Contributing to Akkoma
You wish to add a new feature in Akkoma, but don't know how to proceed? This guide takes you through the various steps of the development and contribution process.
If you're looking for stuff to implement or fix, check the [bug-tracker](https://akkoma.dev/AkkomaGang/akkoma/issues) or [forum](https://meta.akkoma.dev/c/requests/5).
Come say hi to us in the [#akkoma-dev chat room](./../#irc)!
## Akkoma Clients
Akkoma is the back-end. Clients have their own repositories and often separate projects. You can check what clients work with Akkoma [on the clients page](../clients/). If you maintain a working client not listed yet, feel free to make a PR [to these docs](./#docs)!
For resources on APIs and such, check the sidebar of this page.
## Docs
The docs are written in Markdown, including certain extensions, and can be found [in the docs folder of the Akkoma repo](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/docs/). The content itself is stored in the `docs` subdirectory.
## Technology
Akkoma is written in [Elixir](https://elixir-lang.org/) and uses [Postgresql](https://www.postgresql.org/) for database. We use [Git](https://git-scm.com/) for collaboration and tracking code changes. Furthermore it can typically run on [Unix and Unix-like OS'es](https://en.wikipedia.org/wiki/Unix-like). For development, you should use an OS which [can run Akkoma](../installation/debian_based_en/).
It's good to have at least some basic understanding of at least Git and Elixir. If this is completely new for you, there's some [videos explaining Git](https://git-scm.com/doc) and Codeberg has a nice article explaining the typical [pull requests Git flow](https://docs.codeberg.org/collaborating/pull-requests-and-git-flow/). For Elixir, you can follow Elixir's own [Getting Started guide](https://elixir-lang.org/getting-started/introduction.html).
## Setting up a development environment
The best way to start is getting the software to run from source so you can start poking on it. Check out the [guides for setting up an Akkoma instance for development](setting_up_akkoma_dev/#setting-up-a-akkoma-development-environment).
## General overview
### Modules
Akkoma has several modules. There are modules for [uploading](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/pleroma/uploaders), [upload filters](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/pleroma/upload/filter), [translators](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/pleroma/akkoma/translators)... The most famous ones are without a doubt the [MRF policies](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/pleroma/web/activity_pub/mrf). Modules are often self contained and a good way to start with development because you don't have to think about much more than just the module itself. We even have an example on [writing your own MRF policy](/configuration/mrf/#writing-your-own-mrf-policy)!
Another easy entry point is the [mix tasks](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/lib/mix/tasks/pleroma). They too are often self contained and don't need you to go through much of the code.
### Activity Streams/Activity Pub
Akkoma uses Activity Streams for both federation, as well as internal representation. It may be interesting to at least go over the specifications of [Activity Pub](https://www.w3.org/TR/activitypub/), [Activity Streams 2.0](https://www.w3.org/TR/activitystreams-core/), and [Activity Streams Vocabulary](https://www.w3.org/TR/activitystreams-vocabulary/). Note that these are not enough to have a full grasp of how everything works, but should at least give you the basics to understand how messages are passed between and inside Akkoma instances.
## Don't forget
When you make changes, you're expected to create [a Pull Request](https://akkoma.dev/AkkomaGang/akkoma/pulls). You don't have to wait until you finish to create the PR, but please do prefix the title of the PR with "WIP: " for as long as you're still working on it. The sooner you create your PR, the sooner people know what you are working on and the sooner you can get feedback and, if needed, help. You can then simply keep working on it until you are finished.
When doing changes, don't forget to add it to the relevant parts of the [CHANGELOG.md](https://akkoma.dev/AkkomaGang/akkoma/src/branch/develop/CHANGELOG.md).
You're expected to write [tests](https://elixirschool.com/en/lessons/testing/basics). While code is generally stored in the `lib` directory, tests are stored in the `test` directory using a similar folder structure. Feel free to peak at other tests to see how they are done. Obviously tests are expected to pass and properly test the functionality you added. If you feel really confident, you could even try to [write a test first and then write the code needed to make it pass](https://en.wikipedia.org/wiki/Test-driven_development)!
Code is formatted using the default formatter that comes with Elixir. You can format a file with e.g. `mix format /path/to/file.ex`. To check if everything is properly formatted, you can run `mix format --check-formatted`.

View File

@ -5,22 +5,37 @@ Akkoma requires some adjustments from the defaults for running the instance loca
## Installing
1. Install Akkoma as explained in [the docs](../installation/debian_based_en.md), with some exceptions:
* You can use your own fork of the repository and add akkoma as a remote `git remote add akkoma 'https://akkoma.dev/AkkomaGang/akkoma.git'`
* You can skip systemd and nginx and all that stuff
* No need to create a dedicated akkoma user, it's easier to just use your own user
* For the DB you can still choose a dedicated user, the mix tasks set it up for you so it's no extra work for you
* You can use your own fork of the repository and add akkoma as a remote `git remote add akkoma 'https://akkoma.dev/AkkomaGang/akkoma.git'`
* For domain you can use `localhost`
* For the DB you can still choose a dedicated user. The mix tasks sets it up, so it's no extra work for you
* instead of creating a `prod.secret.exs`, create `dev.secret.exs`
* No need to prefix with `MIX_ENV=prod`. We're using dev and that's the default MIX_ENV
* You can skip nginx and systemd
* For front-end, you'll probably want to install and use the develop branch instead of the stable branch. There's no guarantee that the stable branch of the FE will always work on the develop branch of the BE.
2. Change the dev.secret.exs
* Change the FE settings to use the installed branch (see also [Frontend Management](/configuration/frontend_management/))
* Change the scheme in `config :pleroma, Pleroma.Web.Endpoint` to http (see examples below)
* If you want to change other settings, you can do that too
3. You can now start the server `mix phx.server`. Once it's build and started, you can access the instance on `http://<host>:<port>` (e.g.http://localhost:4000 ) and should be able to do everything locally you normaly can.
3. You can now start the server with `mix phx.server`. Once it's build and started, you can access the instance on `http://<host>:<port>` (e.g.http://localhost:4000 ) and should be able to do everything locally you normally can.
Example on how to install pleroma-fe and admin-fe using it's develop branch
```sh
mix pleroma.frontend install pleroma-fe --ref develop
mix pleroma.frontend install admin-fe --ref develop
```
Example config to use the pleroma-fe and admin-fe installed from the develop branch
```elixir
config :pleroma, :frontends,
primary: %{"name" => "pleroma-fe", "ref" => "develop"},
admin: %{"name" => "admin-fe", "ref" => "develop"}
```
Example config to change the scheme to http. Change the port if you want to run on another port.
```elixir
config :pleroma, Pleroma.Web.Endpoint,
url: [host: "localhost", scheme: "http", port: 4000],
config :pleroma, Pleroma.Web.Endpoint,
url: [host: "localhost", scheme: "http", port: 4000],
```
Example config to disable captcha. This makes it a bit easier to create test-users.
@ -94,4 +109,4 @@ Update Akkoma as explained in [the docs](../administration/updating.md). Just ma
## Working on multiple branches
If you develop on a separate branch, it's possible you did migrations that aren't merged into another branch you're working on. If you have multiple things you're working on, it's probably best to set up multiple Akkoma instances each with their own database. If you finished with a branch and want to switch back to develop to start a new branch from there, you can drop the database and recreate the database (e.g. by using `config/setup_db.psql`). The commands to drop and recreate the database can be found in [the docs](../administration/backup.md).
If you develop on a separate branch, it's possible you did migrations that aren't merged into another branch you're working on. In that case, it's probably best to set up multiple Akkoma instances each with their own database. If you finished with a branch and want to switch back to develop to start a new branch from there, you can drop the database and recreate the database (e.g. by using `config/setup_db.psql`). The commands to drop and recreate the database can be found in [the docs](../administration/backup.md).

View File

@ -1,6 +1,6 @@
# Installing on OpenBSD
This guide describes the installation and configuration of akkoma (and the required software to run it) on a single OpenBSD 6.6 server.
This guide describes the installation and configuration of akkoma (and the required software to run it) on a single OpenBSD 7.2 server.
For any additional information regarding commands and configuration files mentioned here, check the man pages [online](https://man.openbsd.org/) or directly on your server with the man command.
@ -12,11 +12,10 @@ For any additional information regarding commands and configuration files mentio
To install them, run the following command (with doas or as root):
```
pkg_add elixir gmake git postgresql-server postgresql-contrib cmake ffmpeg ImageMagick erlang-wx-25
pkg_add elixir gmake git postgresql-server postgresql-contrib cmake ffmpeg erlang-wx libmagic
pkg_add erlang-wx # Choose the latest version as package version when promted
```
(Note that the erlang version may change, it was 25 at the time of writing)
Akkoma requires a reverse proxy, OpenBSD has relayd in base (and is used in this guide) and packages/ports are available for nginx (www/nginx) and apache (www/apache-httpd). Independently of the reverse proxy, [acme-client(1)](https://man.openbsd.org/acme-client) can be used to get a certificate from Let's Encrypt.
#### Optional software
@ -29,32 +28,35 @@ Per [`docs/installation/optional/media_graphics_packages.md`](../installation/op
To install the above:
```
pkg_add ImageMagick ffmpeg p5-Image-ExifTool
pkg_add ffmpeg p5-Image-ExifTool
```
#### Creating the akkoma user
Akkoma will be run by a dedicated user, \_akkoma. Before creating it, insert the following lines in login.conf:
Akkoma will be run by a dedicated user, `_akkoma`. Before creating it, insert the following lines in `/etc/login.conf`:
```
akkoma:\
:datasize-max=1536M:\
:datasize-cur=1536M:\
:openfiles-max=4096
```
This creates a "akkoma" login class and sets higher values than default for datasize and openfiles (see [login.conf(5)](https://man.openbsd.org/login.conf)), this is required to avoid having akkoma crash some time after starting.
This creates a `akkoma` login class and sets higher values than default for datasize and openfiles (see [login.conf(5)](https://man.openbsd.org/login.conf)), this is required to avoid having akkoma crash some time after starting.
Create the \_akkoma user, assign it the akkoma login class and create its home directory (/home/\_akkoma/): `useradd -m -L akkoma _akkoma`
Create the `_akkoma` user, assign it the akkoma login class and create its home directory (`/home/_akkoma/`): `useradd -m -L akkoma _akkoma`
#### Clone akkoma's directory
Enter a shell as the \_akkoma user. As root, run `su _akkoma -;cd`. Then clone the repository with `git clone https://akkoma.dev/AkkomaGang/akkoma.git`. Akkoma is now installed in /home/\_akkoma/akkoma/, it will be configured and started at the end of this guide.
Enter a shell as the `_akkoma` user. As root, run `su _akkoma -;cd`. Then clone the repository with `git clone https://akkoma.dev/AkkomaGang/akkoma.git`. Akkoma is now installed in `/home/_akkoma/akkoma/`, it will be configured and started at the end of this guide.
#### PostgreSQL
Start a shell as the \_postgresql user (as root run `su _postgresql -` then run the `initdb` command to initialize postgresql:
You will need to specify pgdata directory to the default (/var/postgresql/data) with the `-D <path>` and set the user to postgres with the `-U <username>` flag. This can be done as follows:
Create `_postgresql`'s user directory (it hasn't been created yet): `mdir var/postgresql/data`. To set it as home
directory for user `_postgresql` run `usermod -d /var/postgresql/data _postgresql`.
Start a shell as the `_postgresql` user (as root run `su _postgresql -` then run the `initdb` command to initialize postgresql.
You will need to specify pgdata directory to the default (`/var/postgresql/data`) with the `-D <path>` and set the user to postgres with the `-U <username>` flag. This can be done as follows:
```
initdb -D /var/postgresql/data -U postgres
```
If you are not using the default directory, you will have to update the `datadir` variable in the /etc/rc.d/postgresql script.
If you are not using the default directory, you will have to update the `datadir` variable in the `/etc/rc.d/postgresql` script.
When this is done, enable postgresql so that it starts on boot and start it. As root, run:
```
@ -70,7 +72,7 @@ httpd will have three fuctions:
* serve a robots.txt file
* get Let's Encrypt certificates, with acme-client
Insert the following config in httpd.conf:
Insert the following config in `/etc/httpd.conf`:
```
# $OpenBSD: httpd.conf,v 1.17 2017/04/16 08:50:49 ajacoutot Exp $
@ -93,13 +95,10 @@ server "default" {
location "/robots.txt" { root "/htdocs/local/" }
location "/*" { block return 302 "https://$HTTP_HOST$REQUEST_URI" }
}
types {
}
```
Do not forget to change *<IPv4/6 address\>* to your server's address(es). If httpd should only listen on one protocol family, comment one of the two first *listen* options.
Create the /var/www/htdocs/local/ folder and write the content of your robots.txt in /var/www/htdocs/local/robots.txt.
Create the `/var/www/htdocs/local/` folder and write the content of your robots.txt in `/var/www/htdocs/local/robots.txt`.
Check the configuration with `httpd -n`, if it is OK enable and start httpd (as root):
```
rcctl enable httpd
@ -108,7 +107,7 @@ rcctl start httpd
#### acme-client
acme-client is used to get SSL/TLS certificates from Let's Encrypt.
Insert the following configuration in /etc/acme-client.conf:
Insert the following configuration in `/etc/acme-client.conf`:
```
#
# $OpenBSD: acme-client.conf,v 1.4 2017/03/22 11:14:14 benno Exp $
@ -129,7 +128,7 @@ domain <domain name> {
}
```
Replace *<domain name\>* by the domain name you'll use for your instance. As root, run `acme-client -n` to check the config, then `acme-client -ADv <domain name>` to create account and domain keys, and request a certificate for the first time.
Make acme-client run everyday by adding it in /etc/daily.local. As root, run the following command: `echo "acme-client <domain name>" >> /etc/daily.local`.
Make acme-client run everyday by adding it in `/etc/daily.local`. As root, run the following command: `echo "acme-client <domain name>" >> /etc/daily.local`.
Relayd will look for certificates and keys based on the address it listens on (see next part), the easiest way to make them available to relayd is to create a link, as root run:
```
@ -140,7 +139,7 @@ This will have to be done for each IPv4 and IPv6 address relayd listens on.
#### relayd
relayd will be used as the reverse proxy sitting in front of akkoma.
Insert the following configuration in /etc/relayd.conf:
Insert the following configuration in `/etc/relayd.conf`:
```
# $OpenBSD: relayd.conf,v 1.4 2018/03/23 09:55:06 claudio Exp $
@ -198,7 +197,7 @@ rcctl start relayd
#### pf
Enabling and configuring pf is highly recommended.
In /etc/pf.conf, insert the following configuration:
In `/etc/pf.conf`, insert the following configuration:
```
# Macros
if="<network interface>"
@ -222,31 +221,30 @@ pass in quick on $if inet6 proto icmp6 to ($if) icmp6-type { echoreq unreach par
pass in quick on $if proto tcp to ($if) port { http https } # relayd/httpd
pass in quick on $if proto tcp from $authorized_ssh_clients to ($if) port ssh
```
Replace *<network interface\>* by your server's network interface name (which you can get with ifconfig). Consider replacing the content of the authorized\_ssh\_clients macro by, for exemple, your home IP address, to avoid SSH connection attempts from bots.
Replace *<network interface\>* by your server's network interface name (which you can get with ifconfig). Consider replacing the content of the `authorized_ssh_clients` macro by, for example, your home IP address, to avoid SSH connection attempts from bots.
Check pf's configuration by running `pfctl -nf /etc/pf.conf`, load it with `pfctl -f /etc/pf.conf` and enable pf at boot with `rcctl enable pf`.
#### Configure and start akkoma
Enter a shell as \_akkoma (as root `su _akkoma -`) and enter akkoma's installation directory (`cd ~/akkoma/`).
Enter a shell as `_akkoma` (as root `su _akkoma -`) and enter akkoma's installation directory (`cd ~/akkoma/`).
Then follow the main installation guide:
* run `mix deps.get`
* run `MIX_ENV=prod mix pleroma.instance gen` and enter your instance's information when asked
* copy config/generated\_config.exs to config/prod.secret.exs. The default values should be sufficient but you should edit it and check that everything seems OK.
* copy `config/generated_config.exs` to `config/prod.secret.exs`. The default values should be sufficient but you should edit it and check that everything seems OK.
* exit your current shell back to a root one and run `psql -U postgres -f /home/_akkoma/akkoma/config/setup_db.psql` to setup the database.
* return to a \_akkoma shell into akkoma's installation directory (`su _akkoma -;cd ~/akkoma`) and run `MIX_ENV=prod mix ecto.migrate`
* return to a `_akkoma` shell into akkoma's installation directory (`su _akkoma -;cd ~/akkoma`) and run `MIX_ENV=prod mix ecto.migrate`
As \_akkoma in /home/\_akkoma/akkoma, you can now run `LC_ALL=en_US.UTF-8 MIX_ENV=prod mix phx.server` to start your instance.
As `_akkoma` in `/home/_akkoma/akkoma`, you can now run `LC_ALL=en_US.UTF-8 MIX_ENV=prod mix phx.server` to start your instance.
In another SSH session/tmux window, check that it is working properly by running `ftp -MVo - http://127.0.0.1:4000/api/v1/instance`, you should get json output. Double-check that *uri*'s value is your instance's domain name.
##### Starting akkoma at boot
An rc script to automatically start akkoma at boot hasn't been written yet, it can be run in a tmux session (tmux is in base).
#### Create administrative user
If your instance is up and running, you can create your first user with administrative rights with the following command as the \_akkoma user.
If your instance is up and running, you can create your first user with administrative rights with the following command as the `_akkoma` user.
```
LC_ALL=en_US.UTF-8 MIX_ENV=prod mix pleroma.user new <username> <your@emailaddress> --admin
```

View File

@ -5,7 +5,7 @@
This guide covers a installation using an OTP release. To install Akkoma from source, please check out the corresponding guide for your distro.
## Pre-requisites
* A machine running Linux with GNU (e.g. Debian, Ubuntu) or musl (e.g. Alpine) libc and `x86_64`, `aarch64` or `armv7l` CPU, you have root access to. If you are not sure if it's compatible see [Detecting flavour section](#detecting-flavour) below
* A machine running Linux with GNU (e.g. Debian, Ubuntu) or musl (e.g. Alpine) libc and an `x86_64` CPU you have root access to. If you are not sure if it's compatible see [Detecting flavour section](#detecting-flavour) below
* For installing OTP releases on RedHat-based distros like Fedora and Centos Stream, please follow [this guide](./otp_redhat_en.md) instead.
* A (sub)domain pointed to the machine

View File

@ -67,33 +67,92 @@ def run(["prune_objects" | args]) do
OptionParser.parse(
args,
strict: [
vacuum: :boolean
vacuum: :boolean,
keep_threads: :boolean,
keep_non_public: :boolean
]
)
start_pleroma()
deadline = Pleroma.Config.get([:instance, :remote_post_retention_days])
time_deadline = NaiveDateTime.utc_now() |> NaiveDateTime.add(-(deadline * 86_400))
Logger.info("Pruning objects older than #{deadline} days")
log_message = "Pruning objects older than #{deadline} days"
time_deadline =
NaiveDateTime.utc_now()
|> NaiveDateTime.add(-(deadline * 86_400))
log_message =
if Keyword.get(options, :keep_non_public) do
log_message <> ", keeping non public posts"
else
log_message
end
from(o in Object,
where:
fragment(
"?->'to' \\? ? OR ?->'cc' \\? ?",
o.data,
^Pleroma.Constants.as_public(),
o.data,
^Pleroma.Constants.as_public()
),
where: o.inserted_at < ^time_deadline,
where:
log_message =
if Keyword.get(options, :keep_threads) do
log_message <> ", keeping threads intact"
else
log_message
end
Logger.info(log_message)
if Keyword.get(options, :keep_threads) do
# We want to delete objects from threads where
# 1. the newest post is still old
# 2. none of the activities is local
# 3. none of the activities is bookmarked
# 4. optionally none of the posts is non-public
deletable_context =
if Keyword.get(options, :keep_non_public) do
Pleroma.Activity
|> join(:left, [a], b in Pleroma.Bookmark, on: a.id == b.activity_id)
|> group_by([a], fragment("? ->> 'context'::text", a.data))
|> having(
[a],
not fragment(
# Posts (checked on Create Activity) is non-public
"bool_or((not(?->'to' \\? ? OR ?->'cc' \\? ?)) and ? ->> 'type' = 'Create')",
a.data,
^Pleroma.Constants.as_public(),
a.data,
^Pleroma.Constants.as_public(),
a.data
)
)
else
Pleroma.Activity
|> join(:left, [a], b in Pleroma.Bookmark, on: a.id == b.activity_id)
|> group_by([a], fragment("? ->> 'context'::text", a.data))
end
|> having([a], max(a.updated_at) < ^time_deadline)
|> having([a], not fragment("bool_or(?)", a.local))
|> having([_, b], fragment("max(?::text) is null", b.id))
|> select([a], fragment("? ->> 'context'::text", a.data))
Pleroma.Object
|> where([o], fragment("? ->> 'context'::text", o.data) in subquery(deletable_context))
else
if Keyword.get(options, :keep_non_public) do
Pleroma.Object
|> where(
[o],
fragment(
"?->'to' \\? ? OR ?->'cc' \\? ?",
o.data,
^Pleroma.Constants.as_public(),
o.data,
^Pleroma.Constants.as_public()
)
)
else
Pleroma.Object
end
|> where([o], o.updated_at < ^time_deadline)
|> where(
[o],
fragment("split_part(?->>'actor', '/', 3) != ?", o.data, ^Pleroma.Web.Endpoint.host())
)
)
end
|> Repo.delete_all(timeout: :infinity)
prune_hashtags_query = """

View File

@ -121,7 +121,7 @@ def user_invitation_email(
"user invitation email body",
"""
<h3>You are invited to %{instance_name}</h3>
<p>%{inviter_name} invites you to join %{instance_name}, an instance of Pleroma federated social networking platform.</p>
<p>%{inviter_name} invites you to join %{instance_name}, an instance of Akkoma federated social networking platform.</p>
<p>Click the following link to register: <a href="%{registration_url}">accept invitation</a>.</p>
""",
instance_name: instance_name(),
@ -357,7 +357,7 @@ def backup_is_ready_email(backup, admin_user_id \\ nil) do
"static_pages",
"account archive email body - self-requested",
"""
<p>You requested a full backup of your Pleroma account. It's ready for download:</p>
<p>You requested a full backup of your Akkoma account. It's ready for download:</p>
<p><a href="%{download_url}">%{download_url}</a></p>
""",
download_url: download_url
@ -369,7 +369,7 @@ def backup_is_ready_email(backup, admin_user_id \\ nil) do
"static_pages",
"account archive email body - admin requested",
"""
<p>Admin @%{admin_nickname} requested a full backup of your Pleroma account. It's ready for download:</p>
<p>Admin @%{admin_nickname} requested a full backup of your Akkoma account. It's ready for download:</p>
<p><a href="%{download_url}">%{download_url}</a></p>
""",
admin_nickname: admin.nickname,

View File

@ -252,7 +252,7 @@ def download(name, url, as) do
with :ok <- validate_shareable_packs_available(uri),
{:ok, remote_pack} <-
uri |> URI.merge("/api/v1/pleroma/emoji/pack?name=#{name}") |> http_get(),
uri |> URI.merge("/api/v1/pleroma/emoji/pack?name=#{URI.encode(name)}") |> http_get(),
{:ok, %{sha: sha, url: url} = pack_info} <- fetch_pack_info(remote_pack, uri, name),
{:ok, archive} <- download_archive(url, sha),
pack <- copy_as(remote_pack, as || name),
@ -593,7 +593,9 @@ defp fetch_pack_info(remote_pack, uri, name) do
{:ok,
%{
sha: sha,
url: URI.merge(uri, "/api/v1/pleroma/emoji/packs/archive?name=#{name}") |> to_string()
url:
URI.merge(uri, "/api/v1/pleroma/emoji/packs/archive?name=#{URI.encode(name)}")
|> to_string()
}}
%{"fallback-src" => src, "fallback-src-sha256" => sha} when is_binary(src) ->

View File

@ -155,14 +155,13 @@ def following_count(%User{} = user) do
|> Repo.aggregate(:count, :id)
end
def get_follow_requests(%User{id: id}) do
def get_follow_requests_query(%User{id: id}) do
__MODULE__
|> join(:inner, [r], f in assoc(r, :follower))
|> join(:inner, [r], f in assoc(r, :follower), as: :follower)
|> where([r], r.state == ^:follow_pending)
|> where([r], r.following_id == ^id)
|> where([r, f], f.is_active == true)
|> select([r, f], f)
|> Repo.all()
|> where([r, follower: f], f.is_active == true)
|> select([r, follower: f], f)
end
def following?(%User{id: follower_id}, %User{id: followed_id}) do

11
lib/pleroma/iso639.ex Normal file
View File

@ -0,0 +1,11 @@
defmodule Pleroma.ISO639 do
@file "priv/language-codes.json"
@data File.read!(@file)
|> Jason.decode!()
for %{"alpha2" => alpha2} <- @data do
def valid_alpha2?(unquote(alpha2)), do: true
end
def valid_alpha2?(_alpha2), do: false
end

View File

@ -14,6 +14,8 @@ defmodule Pleroma.Upload.Filter.Exiftool do
# Formats not compatible with exiftool at this time
def filter(%Pleroma.Upload{content_type: "image/heic"}), do: {:ok, :noop}
def filter(%Pleroma.Upload{content_type: "image/webp"}), do: {:ok, :noop}
def filter(%Pleroma.Upload{content_type: "image/svg+xml"}), do: {:ok, :noop}
def filter(%Pleroma.Upload{content_type: "image/jxl"}), do: {:ok, :noop}
def filter(%Pleroma.Upload{tempfile: file, content_type: "image" <> _}) do
try do

View File

@ -285,7 +285,13 @@ def cached_muted_users_ap_ids(user) do
defdelegate following(user), to: FollowingRelationship
defdelegate following?(follower, followed), to: FollowingRelationship
defdelegate following_ap_ids(user), to: FollowingRelationship
defdelegate get_follow_requests(user), to: FollowingRelationship
defdelegate get_follow_requests_query(user), to: FollowingRelationship
def get_follow_requests(user) do
get_follow_requests_query(user)
|> Repo.all()
end
defdelegate search(query, opts \\ []), to: User.Search
@doc """
@ -2020,6 +2026,7 @@ defp create_service_actor(uri, nickname) do
%User{
invisible: true,
local: true,
actor_type: "Application",
ap_id: uri,
nickname: nickname,
follower_address: uri <> "/followers"

View File

@ -155,10 +155,17 @@ defp get_policies(_), do: []
# - https://extra.baddomain.net/
# Does NOT match the following:
# - https://maybebaddomain.net/
# *.baddomain.net
def subdomain_regex("*." <> domain), do: subdomain_regex(domain)
# baddomain.net
def subdomain_regex(domain) do
~r/^(.+\.)?#{Regex.escape(domain)}$/i
if String.ends_with?(domain, ".*") do
~r/^(.+\.)?#{Regex.escape(String.replace_suffix(domain, ".*", ""))}\.(.+)$/i
else
~r/^(.+\.)?#{Regex.escape(domain)}$/i
end
end
@spec subdomains_regex([String.t()]) :: [Regex.t()]

View File

@ -30,6 +30,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.ArticleNotePageValidator do
field(:replies, {:array, ObjectValidators.ObjectID}, default: [])
field(:source, :map)
field(:contentMap, :map)
end
def cast_and_apply(data) do
@ -146,6 +147,21 @@ defp fix_source(%{"source" => source} = object) when is_binary(source) do
defp fix_source(object), do: object
defp fix_content_map_languages(%{"contentMap" => content_map} = object)
when is_map(content_map) do
# Only allow valid languages
content_map =
content_map
|> Enum.reject(fn {lang, _content} ->
!Pleroma.ISO639.valid_alpha2?(lang)
end)
|> Enum.into(%{})
Map.put(object, "contentMap", content_map)
end
defp fix_content_map_languages(object), do: object
defp fix(data) do
data
|> CommonFixes.fix_actor()
@ -158,6 +174,7 @@ defp fix(data) do
|> Transmogrifier.fix_attachments()
|> Transmogrifier.fix_emoji()
|> Transmogrifier.fix_content_map()
|> fix_content_map_languages()
end
def changeset(struct, data) do

View File

@ -346,11 +346,16 @@ def fix_tag(%{"tag" => %{} = tag} = object) do
def fix_tag(object), do: object
# content map usually only has one language so this will do for now.
def fix_content_map(%{"contentMap" => content_map} = object) do
def fix_content_map(%{"contentMap" => content_map} = object) when is_map(content_map) do
content_groups = Map.to_list(content_map)
{_, content} = Enum.at(content_groups, 0)
Map.put(object, "content", content)
if Enum.empty?(content_groups) do
object
else
{_, content} = Enum.at(content_groups, 0)
Map.put(object, "content", content)
end
end
def fix_content_map(object), do: object

View File

@ -19,6 +19,7 @@ def index_operation do
summary: "Retrieve follow requests",
security: [%{"oAuth" => ["read:follows", "follow"]}],
operationId: "FollowRequestController.index",
parameters: pagination_params(),
responses: %{
200 =>
Operation.response("Array of Account", "application/json", %Schema{
@ -62,4 +63,22 @@ defp id_param do
required: true
)
end
defp pagination_params do
[
Operation.parameter(:max_id, :query, :string, "Return items older than this ID"),
Operation.parameter(
:since_id,
:query,
:string,
"Return the oldest items newer than this ID"
),
Operation.parameter(
:limit,
:query,
%Schema{type: :integer, default: 20},
"Maximum number of items to return. Will be ignored if it's more than 40"
)
]
end
end

View File

@ -22,6 +22,8 @@ defmodule Pleroma.Web.CommonAPI.ActivityDraft do
attachments: [],
in_reply_to: nil,
in_reply_to_conversation: nil,
language: nil,
content_map: %{},
quote_id: nil,
quote: nil,
visibility: nil,
@ -58,6 +60,7 @@ def create(user, params) do
|> with_valid(&visibility/1)
|> with_valid(&quote_id/1)
|> content()
|> with_valid(&language/1)
|> with_valid(&to_and_cc/1)
|> with_valid(&context/1)
|> sensitive()
@ -133,6 +136,20 @@ defp quote_id(%{params: %{quote_id: %Activity{} = quote}} = draft) do
defp quote_id(draft), do: draft
defp language(%{params: %{language: language}, content_html: content} = draft)
when is_binary(language) do
if Pleroma.ISO639.valid_alpha2?(language) do
%__MODULE__{draft | content_map: %{language => content}}
else
add_error(draft, dgettext("errors", "Invalid language"))
end
end
defp language(%{content_html: content} = draft) do
# Use a default language if no language is specified
%__MODULE__{draft | content_map: %{"en" => content}}
end
defp visibility(%{params: params} = draft) do
case CommonAPI.get_visibility(params, draft.in_reply_to, draft.in_reply_to_conversation) do
{visibility, "direct"} when visibility != "direct" ->
@ -224,6 +241,7 @@ defp object(draft) do
"mediaType" => Utils.get_content_type(draft.params[:content_type])
})
|> Map.put("generator", draft.params[:generator])
|> Map.put("contentMap", draft.content_map)
%__MODULE__{draft | object: object}
end

View File

@ -328,20 +328,27 @@ def date_to_asctime(date) do
end
def to_masto_date(%NaiveDateTime{} = date) do
date
|> NaiveDateTime.to_iso8601()
|> String.replace(~r/(\.\d+)?$/, ".000Z", global: false)
# NOTE: Elixirs ISO 8601 format is a superset of the real standard
# It supports negative years for example.
# ISO8601 only supports years before 1583 with mutual agreement
if date.year < 1583 do
"1970-01-01T00:00:00Z"
else
date
|> NaiveDateTime.to_iso8601()
|> String.replace(~r/(\.\d+)?$/, ".000Z", global: false)
end
end
def to_masto_date(date) when is_binary(date) do
with {:ok, date} <- NaiveDateTime.from_iso8601(date) do
to_masto_date(date)
else
_ -> ""
_ -> "1970-01-01T00:00:00Z"
end
end
def to_masto_date(_), do: ""
def to_masto_date(_), do: "1970-01-01T00:00:00Z"
defp shortname(name) do
with max_length when max_length > 0 <-

View File

@ -5,9 +5,13 @@
defmodule Pleroma.Web.MastodonAPI.FollowRequestController do
use Pleroma.Web, :controller
import Pleroma.Web.ControllerHelper,
only: [add_link_headers: 2]
alias Pleroma.User
alias Pleroma.Web.CommonAPI
alias Pleroma.Web.Plugs.OAuthScopesPlug
alias Pleroma.Pagination
plug(Pleroma.Web.ApiSpec.CastAndValidate)
plug(:assign_follower when action != :index)
@ -24,10 +28,15 @@ defmodule Pleroma.Web.MastodonAPI.FollowRequestController do
defdelegate open_api_operation(action), to: Pleroma.Web.ApiSpec.FollowRequestOperation
@doc "GET /api/v1/follow_requests"
def index(%{assigns: %{user: followed}} = conn, _params) do
follow_requests = User.get_follow_requests(followed)
def index(%{assigns: %{user: followed}} = conn, params) do
follow_requests =
followed
|> User.get_follow_requests_query()
|> Pagination.fetch_paginated(params, :keyset, :follower)
render(conn, "index.json", for: followed, users: follow_requests, as: :user)
conn
|> add_link_headers(follow_requests)
|> render("index.json", for: followed, users: follow_requests, as: :user)
end
@doc "POST /api/v1/follow_requests/:id/authorize"

View File

@ -336,7 +336,8 @@ defp maybe_put_follow_requests_count(
%User{id: user_id}
) do
count =
User.get_follow_requests(user)
user
|> User.get_follow_requests()
|> length()
data

View File

@ -169,6 +169,7 @@ def render(
|> Enum.map(fn user -> AccountView.render("mention.json", %{user: user}) end)
{pinned?, pinned_at} = pin_data(object, user)
lang = language(object)
%{
id: to_string(activity.id),
@ -199,7 +200,7 @@ def render(
mentions: mentions,
tags: reblogged[:tags] || [],
application: build_application(object.data["generator"]),
language: nil,
language: lang,
emojis: [],
pleroma: %{
local: activity.local,
@ -357,6 +358,7 @@ def render("show.json", %{activity: %{data: %{"object" => _object}} = activity}
{pinned?, pinned_at} = pin_data(object, user)
quote = Activity.get_quoted_activity_from_object(object)
lang = language(object)
%{
id: to_string(activity.id),
@ -391,7 +393,7 @@ def render("show.json", %{activity: %{data: %{"object" => _object}} = activity}
mentions: mentions,
tags: build_tags(tags),
application: build_application(object.data["generator"]),
language: nil,
language: lang,
emojis: build_emojis(object.data["emoji"]),
quote_id: if(quote, do: quote.id, else: nil),
quote: maybe_render_quote(quote, opts),
@ -784,4 +786,12 @@ defp get_source_content_type(%{"mediaType" => type} = _source) do
defp get_source_content_type(_source) do
Utils.get_content_type(nil)
end
defp language(%Object{data: %{"contentMap" => contentMap}}) when is_map(contentMap) do
contentMap
|> Map.keys()
|> Enum.at(0)
end
defp language(_), do: nil
end

View File

@ -42,7 +42,7 @@ def maybe_put_rel_me("http" <> _ = target_page, profile_urls) when is_list(profi
"me"
rescue
e -> nil
_ -> nil
end
def maybe_put_rel_me(_, _) do

View File

@ -25,7 +25,7 @@ def show(%{assigns: %{notice_id: notice_id}} = conn, _params) do
true <- Visibility.is_public?(activity.object),
{_, true} <- {:visible?, Visibility.visible_for_user?(activity, _reading_user = nil)},
%User{} = user <- User.get_by_ap_id(activity.object.data["actor"]) do
meta = Metadata.build_tags(%{activity_id: notice_id, object: activity.object, user: user})
meta = Metadata.build_tags(%{url: activity.data["id"], object: activity.object, user: user})
timeline =
activity.object.data["context"]

View File

@ -18,10 +18,21 @@ defmodule Pleroma.Web.TwitterAPI.Controller do
action_fallback(:errors)
def confirm_email(conn, %{"user_id" => uid, "token" => token}) do
with %User{} = user <- User.get_cached_by_id(uid),
true <- user.local and !user.is_confirmed and user.confirmation_token == token,
{:ok, _} <- User.confirm(user) do
redirect(conn, to: "/")
case User.get_cached_by_id(uid) do
%User{local: true, is_confirmed: false, confirmation_token: ^token} = user ->
case User.confirm(user) do
{:ok, _} ->
redirect(conn, to: "/")
{:error, _} ->
json_reply(conn, 400, "Unable to confirm")
end
%User{is_confirmed: true} ->
json_reply(conn, 400, "Already verified email")
_ ->
json_reply(conn, 400, "Couldn't verify email")
end
end

View File

@ -4,7 +4,7 @@ defmodule Pleroma.Mixfile do
def project do
[
app: :pleroma,
version: version("3.5.0"),
version: version("3.6.0"),
elixir: "~> 1.12",
elixirc_paths: elixirc_paths(Mix.env()),
compilers: [:phoenix] ++ Mix.compilers(),
@ -201,7 +201,7 @@ defp deps do
ref: "1c1b99ea41a457761383d81aaf6a606913996fe7",
only: [:dev, :test],
runtime: false},
{:mock, "~> 0.3.5", only: :test},
{:mock, "~> 0.3.7", only: :test},
{:excoveralls, "0.15.1", only: :test},
{:mox, "~> 1.0", only: :test},
{:websockex, "~> 0.4.3", only: :test},

View File

@ -6,7 +6,7 @@
"benchee": {:hex, :benchee, "1.1.0", "f3a43817209a92a1fade36ef36b86e1052627fd8934a8b937ac9ab3a76c43062", [:mix], [{:deep_merge, "~> 1.0", [hex: :deep_merge, repo: "hexpm", optional: false]}, {:statistex, "~> 1.0", [hex: :statistex, repo: "hexpm", optional: false]}], "hexpm", "7da57d545003165a012b587077f6ba90b89210fd88074ce3c60ce239eb5e6d93"},
"bunt": {:hex, :bunt, "0.2.1", "e2d4792f7bc0ced7583ab54922808919518d0e57ee162901a16a1b6664ef3b14", [:mix], [], "hexpm", "a330bfb4245239787b15005e66ae6845c9cd524a288f0d141c148b02603777a5"},
"cachex": {:hex, :cachex, "3.5.0", "f715390a9e93125980187dcd7c4036ece92d273fbd9ec009a8ffa480abdc51f8", [:mix], [{:eternal, "~> 1.2", [hex: :eternal, repo: "hexpm", optional: false]}, {:jumper, "~> 1.0", [hex: :jumper, repo: "hexpm", optional: false]}, {:sleeplocks, "~> 1.1", [hex: :sleeplocks, repo: "hexpm", optional: false]}, {:unsafe, "~> 1.0", [hex: :unsafe, repo: "hexpm", optional: false]}], "hexpm", "fac2ebfa200dd9ffba08cdcef404426ccadfcb92281ca34f810535712d02b049"},
"calendar": {:hex, :calendar, "1.0.0", "f52073a708528482ec33d0a171954ca610fe2bd28f1e871f247dc7f1565fa807", [:mix], [{:tzdata, "~> 0.5.20 or ~> 0.1.201603 or ~> 1.0", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm", "990e9581920c82912a5ee50e62ff5ef96da6b15949a2ee4734f935fdef0f0a6f"},
"calendar": {:hex, :calendar, "1.0.0", "f52073a708528482ec33d0a171954ca610fe2bd28f1e871f247dc7f1565fa807", [:mix], [{:tzdata, "~> 0.1.201603 or ~> 0.5.20 or ~> 1.0", [hex: :tzdata, repo: "hexpm", optional: false]}], "hexpm", "990e9581920c82912a5ee50e62ff5ef96da6b15949a2ee4734f935fdef0f0a6f"},
"captcha": {:git, "https://git.pleroma.social/pleroma/elixir-libraries/elixir-captcha.git", "e0f16822d578866e186a0974d65ad58cddc1e2ab", [ref: "e0f16822d578866e186a0974d65ad58cddc1e2ab"]},
"castore": {:hex, :castore, "0.1.20", "62a0126cbb7cb3e259257827b9190f88316eb7aa3fdac01fd6f2dfd64e7f46e9", [:mix], [], "hexpm", "a020b7650529c986c454a4035b6b13a328e288466986307bea3aadb4c95ac98a"},
"certifi": {:hex, :certifi, "2.9.0", "6f2a475689dd47f19fb74334859d460a2dc4e3252a3324bd2111b8f0429e7e21", [:rebar3], [], "hexpm", "266da46bdb06d6c6d35fde799bcb28d36d985d424ad7c08b5bb48f5b5cdd4641"},
@ -30,7 +30,7 @@
"ecto": {:hex, :ecto, "3.9.4", "3ee68e25dbe0c36f980f1ba5dd41ee0d3eb0873bccae8aeaf1a2647242bffa35", [:mix], [{:decimal, "~> 1.6 or ~> 2.0", [hex: :decimal, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "de5f988c142a3aa4ec18b85a4ec34a2390b65b24f02385c1144252ff6ff8ee75"},
"ecto_enum": {:hex, :ecto_enum, "1.4.0", "d14b00e04b974afc69c251632d1e49594d899067ee2b376277efd8233027aec8", [:mix], [{:ecto, ">= 3.0.0", [hex: :ecto, repo: "hexpm", optional: false]}, {:ecto_sql, "> 3.0.0", [hex: :ecto_sql, repo: "hexpm", optional: false]}, {:mariaex, ">= 0.0.0", [hex: :mariaex, repo: "hexpm", optional: true]}, {:postgrex, ">= 0.0.0", [hex: :postgrex, repo: "hexpm", optional: true]}], "hexpm", "8fb55c087181c2b15eee406519dc22578fa60dd82c088be376d0010172764ee4"},
"ecto_psql_extras": {:hex, :ecto_psql_extras, "0.7.10", "e14d400930f401ca9f541b3349212634e44027d7f919bbb71224d7ac0d0e8acd", [:mix], [{:ecto_sql, "~> 3.4", [hex: :ecto_sql, repo: "hexpm", optional: false]}, {:postgrex, "~> 0.15.7 or ~> 0.16.0", [hex: :postgrex, repo: "hexpm", optional: false]}, {:table_rex, "~> 3.1.1", [hex: :table_rex, repo: "hexpm", optional: false]}], "hexpm", "505e8cd81e4f17c090be0f99e92b1b3f0fd915f98e76965130b8ccfb891e7088"},
"ecto_sql": {:hex, :ecto_sql, "3.9.2", "34227501abe92dba10d9c3495ab6770e75e79b836d114c41108a4bf2ce200ad5", [:mix], [{:db_connection, "~> 2.5 or ~> 2.4.1", [hex: :db_connection, repo: "hexpm", optional: false]}, {:ecto, "~> 3.9.2", [hex: :ecto, repo: "hexpm", optional: false]}, {:myxql, "~> 0.6.0", [hex: :myxql, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.16.0 or ~> 1.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:tds, "~> 2.1.1 or ~> 2.2", [hex: :tds, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4.0 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "1eb5eeb4358fdbcd42eac11c1fbd87e3affd7904e639d77903c1358b2abd3f70"},
"ecto_sql": {:hex, :ecto_sql, "3.9.2", "34227501abe92dba10d9c3495ab6770e75e79b836d114c41108a4bf2ce200ad5", [:mix], [{:db_connection, "~> 2.4.1 or ~> 2.5", [hex: :db_connection, repo: "hexpm", optional: false]}, {:ecto, "~> 3.9.2", [hex: :ecto, repo: "hexpm", optional: false]}, {:myxql, "~> 0.6.0", [hex: :myxql, repo: "hexpm", optional: true]}, {:postgrex, "~> 0.16.0 or ~> 1.0", [hex: :postgrex, repo: "hexpm", optional: true]}, {:tds, "~> 2.1.1 or ~> 2.2", [hex: :tds, repo: "hexpm", optional: true]}, {:telemetry, "~> 0.4.0 or ~> 1.0", [hex: :telemetry, repo: "hexpm", optional: false]}], "hexpm", "1eb5eeb4358fdbcd42eac11c1fbd87e3affd7904e639d77903c1358b2abd3f70"},
"elasticsearch": {:git, "https://akkoma.dev/AkkomaGang/elasticsearch-elixir.git", "6cd946f75f6ab9042521a009d1d32d29a90113ca", [ref: "main"]},
"elixir_make": {:hex, :elixir_make, "0.6.3", "bc07d53221216838d79e03a8019d0839786703129599e9619f4ab74c8c096eac", [:mix], [], "hexpm", "f5cbd651c5678bcaabdbb7857658ee106b12509cd976c2c2fca99688e1daf716"},
"erlex": {:hex, :erlex, "0.2.6", "c7987d15e899c7a2f34f5420d2a2ea0d659682c06ac607572df55a43753aa12e", [:mix], [], "hexpm", "2ed2e25711feb44d52b17d2780eabf998452f6efda104877a3881c2f8c0c0c75"},

1
priv/language-codes.json Normal file

File diff suppressed because one or more lines are too long

View File

@ -0,0 +1,21 @@
defmodule Pleroma.Repo.Migrations.InstanceActorsToActorTypeApplication do
use Ecto.Migration
def up do
execute("""
update users
set actor_type = 'Application'
where local
and (ap_id like '%/relay' or ap_id like '%/internal/fetch')
""")
end
def down do
execute("""
update users
set actor_type = 'Person'
where local
and (ap_id like '%/relay' or ap_id like '%/internal/fetch')
""")
end
end

View File

@ -41,7 +41,11 @@
"@type": "@id"
},
"vcard": "http://www.w3.org/2006/vcard/ns#",
"formerRepresentations": "litepub:formerRepresentations"
"formerRepresentations": "litepub:formerRepresentations",
"contentMap": {
"@id": "as:content",
"@container": "@language"
}
}
]
}

View File

@ -0,0 +1,38 @@
{
"@context": [
"https://www.w3.org/ns/activitystreams",
{
"ostatus": "http://ostatus.org#",
"atomUri": "ostatus:atomUri",
"inReplyToAtomUri": "ostatus:inReplyToAtomUri",
"conversation": "ostatus:conversation",
"sensitive": "as:sensitive",
"toot": "http://joinmastodon.org/ns#",
"votersCount": "toot:votersCount"
}
],
"id": "https://mastodon.social/users/akkoma_ap_integration_tester/statuses/109671288784583764",
"type": "Note",
"summary": null,
"inReplyTo": null,
"published": "2023-01-11T15:31:01Z",
"url": "https://mastodon.social/@akkoma_ap_integration_tester/109671288784583764",
"attributedTo": "https://mastodon.social/users/akkoma_ap_integration_tester",
"to": [
"https://www.w3.org/ns/activitystreams#Public"
],
"cc": [
"https://mastodon.social/users/akkoma_ap_integration_tester/followers"
],
"sensitive": false,
"atomUri": "https://mastodon.social/users/akkoma_ap_integration_tester/statuses/109671288784583764",
"inReplyToAtomUri": null,
"conversation": "tag:mastodon.social,2023-01-11:objectId=376794415:objectType=Conversation",
"content": "<p>tag</p>",
"contentMap": {
"ja": "<p>tag</p>"
},
"attachment": [],
"tag": [],
"replies": []
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 95 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 95 B

View File

@ -0,0 +1,12 @@
{
"files": {
"blank": "blank.png",
"blank2": "blank2.png"
},
"pack": {
"description": "Test description",
"homepage": "https://pleroma.social",
"license": "Test license",
"share-files": true
}
}

View File

@ -46,7 +46,6 @@ test "it replaces objects with references" do
describe "prune_objects" do
test "it prunes old objects from the database" do
insert(:note)
deadline = Pleroma.Config.get([:instance, :remote_post_retention_days]) + 1
date =
@ -55,18 +54,304 @@ test "it prunes old objects from the database" do
|> Timex.to_naive_datetime()
|> NaiveDateTime.truncate(:second)
%{id: id} =
insert(:note)
%{id: note_remote_public_id} =
:note
|> insert()
|> Ecto.Changeset.change(%{inserted_at: date})
|> Ecto.Changeset.change(%{updated_at: date})
|> Repo.update!()
assert length(Repo.all(Object)) == 2
note_remote_non_public =
%{id: note_remote_non_public_id, data: note_remote_non_public_data} =
:note
|> insert()
note_remote_non_public
|> Ecto.Changeset.change(%{
updated_at: date,
data: note_remote_non_public_data |> update_in(["to"], fn _ -> [] end)
})
|> Repo.update!()
assert length(Repo.all(Object)) == 3
Mix.Tasks.Pleroma.Database.run(["prune_objects"])
assert length(Repo.all(Object)) == 1
refute Object.get_by_id(id)
refute Object.get_by_id(note_remote_public_id)
refute Object.get_by_id(note_remote_non_public_id)
end
test "with the --keep-non-public option it still keeps non-public posts even if they are not local" do
deadline = Pleroma.Config.get([:instance, :remote_post_retention_days]) + 1
date =
Timex.now()
|> Timex.shift(days: -deadline)
|> Timex.to_naive_datetime()
|> NaiveDateTime.truncate(:second)
insert(:note)
%{id: note_remote_id} =
:note
|> insert()
|> Ecto.Changeset.change(%{updated_at: date})
|> Repo.update!()
note_remote_non_public =
%{data: note_remote_non_public_data} =
:note
|> insert()
note_remote_non_public
|> Ecto.Changeset.change(%{
updated_at: date,
data: note_remote_non_public_data |> update_in(["to"], fn _ -> [] end)
})
|> Repo.update!()
assert length(Repo.all(Object)) == 3
Mix.Tasks.Pleroma.Database.run(["prune_objects", "--keep-non-public"])
assert length(Repo.all(Object)) == 2
refute Object.get_by_id(note_remote_id)
end
test "with the --keep-threads and --keep-non-public option it keeps old threads with non-public replies even if the interaction is not local" do
# For non-public we only check Create Activities because only these are relevant for threads
# Flags are always non-public, Announces from relays can be non-public...
deadline = Pleroma.Config.get([:instance, :remote_post_retention_days]) + 1
old_insert_date =
Timex.now()
|> Timex.shift(days: -deadline)
|> Timex.to_naive_datetime()
|> NaiveDateTime.truncate(:second)
remote_user1 = insert(:user, local: false)
remote_user2 = insert(:user, local: false)
# Old remote non-public reply (should be kept)
{:ok, old_remote_post1_activity} =
CommonAPI.post(remote_user1, %{status: "some thing", local: false})
old_remote_post1_activity
|> Ecto.Changeset.change(%{local: false, updated_at: old_insert_date})
|> Repo.update!()
{:ok, old_remote_non_public_reply_activity} =
CommonAPI.post(remote_user2, %{
status: "some reply",
in_reply_to_status_id: old_remote_post1_activity.id
})
old_remote_non_public_reply_activity
|> Ecto.Changeset.change(%{
local: false,
updated_at: old_insert_date,
data: old_remote_non_public_reply_activity.data |> update_in(["to"], fn _ -> [] end)
})
|> Repo.update!()
# Old remote non-public Announce (should be removed)
{:ok, old_remote_post2_activity = %{data: %{"object" => old_remote_post2_id}}} =
CommonAPI.post(remote_user1, %{status: "some thing", local: false})
old_remote_post2_activity
|> Ecto.Changeset.change(%{local: false, updated_at: old_insert_date})
|> Repo.update!()
{:ok, old_remote_non_public_repeat_activity} =
CommonAPI.repeat(old_remote_post2_activity.id, remote_user2)
old_remote_non_public_repeat_activity
|> Ecto.Changeset.change(%{
local: false,
updated_at: old_insert_date,
data: old_remote_non_public_repeat_activity.data |> update_in(["to"], fn _ -> [] end)
})
|> Repo.update!()
assert length(Repo.all(Object)) == 3
Mix.Tasks.Pleroma.Database.run(["prune_objects", "--keep-threads", "--keep-non-public"])
Repo.all(Pleroma.Activity)
assert length(Repo.all(Object)) == 2
refute Object.get_by_ap_id(old_remote_post2_id)
end
test "with the --keep-threads option it still keeps non-old threads even with no local interactions" do
remote_user = insert(:user, local: false)
remote_user2 = insert(:user, local: false)
{:ok, remote_post_activity} =
CommonAPI.post(remote_user, %{status: "some thing", local: false})
{:ok, remote_post_reply_activity} =
CommonAPI.post(remote_user2, %{
status: "some reply",
in_reply_to_status_id: remote_post_activity.id
})
remote_post_activity
|> Ecto.Changeset.change(%{local: false})
|> Repo.update!()
remote_post_reply_activity
|> Ecto.Changeset.change(%{local: false})
|> Repo.update!()
assert length(Repo.all(Object)) == 2
Mix.Tasks.Pleroma.Database.run(["prune_objects", "--keep-threads"])
assert length(Repo.all(Object)) == 2
end
test "with the --keep-threads option it deletes old threads with no local interaction" do
deadline = Pleroma.Config.get([:instance, :remote_post_retention_days]) + 1
old_insert_date =
Timex.now()
|> Timex.shift(days: -deadline)
|> Timex.to_naive_datetime()
|> NaiveDateTime.truncate(:second)
remote_user = insert(:user, local: false)
remote_user2 = insert(:user, local: false)
{:ok, old_remote_post_activity} =
CommonAPI.post(remote_user, %{status: "some thing", local: false})
old_remote_post_activity
|> Ecto.Changeset.change(%{local: false, updated_at: old_insert_date})
|> Repo.update!()
{:ok, old_remote_post_reply_activity} =
CommonAPI.post(remote_user2, %{
status: "some reply",
in_reply_to_status_id: old_remote_post_activity.id
})
old_remote_post_reply_activity
|> Ecto.Changeset.change(%{local: false, updated_at: old_insert_date})
|> Repo.update!()
{:ok, old_favourite_activity} =
CommonAPI.favorite(remote_user2, old_remote_post_activity.id)
old_favourite_activity
|> Ecto.Changeset.change(%{local: false, updated_at: old_insert_date})
|> Repo.update!()
{:ok, old_repeat_activity} = CommonAPI.repeat(old_remote_post_activity.id, remote_user2)
old_repeat_activity
|> Ecto.Changeset.change(%{local: false, updated_at: old_insert_date})
|> Repo.update!()
assert length(Repo.all(Object)) == 2
Mix.Tasks.Pleroma.Database.run(["prune_objects", "--keep-threads"])
assert length(Repo.all(Object)) == 0
end
test "with the --keep-threads option it keeps old threads with local interaction" do
deadline = Pleroma.Config.get([:instance, :remote_post_retention_days]) + 1
old_insert_date =
Timex.now()
|> Timex.shift(days: -deadline)
|> Timex.to_naive_datetime()
|> NaiveDateTime.truncate(:second)
remote_user = insert(:user, local: false)
local_user = insert(:user, local: true)
# local reply
{:ok, old_remote_post1_activity} =
CommonAPI.post(remote_user, %{status: "some thing", local: false})
old_remote_post1_activity
|> Ecto.Changeset.change(%{local: false, updated_at: old_insert_date})
|> Repo.update!()
{:ok, old_local_post2_reply_activity} =
CommonAPI.post(local_user, %{
status: "some reply",
in_reply_to_status_id: old_remote_post1_activity.id
})
old_local_post2_reply_activity
|> Ecto.Changeset.change(%{local: true, updated_at: old_insert_date})
|> Repo.update!()
# local Like
{:ok, old_remote_post3_activity} =
CommonAPI.post(remote_user, %{status: "some thing", local: false})
old_remote_post3_activity
|> Ecto.Changeset.change(%{local: false, updated_at: old_insert_date})
|> Repo.update!()
{:ok, old_favourite_activity} = CommonAPI.favorite(local_user, old_remote_post3_activity.id)
old_favourite_activity
|> Ecto.Changeset.change(%{local: true, updated_at: old_insert_date})
|> Repo.update!()
# local Announce
{:ok, old_remote_post4_activity} =
CommonAPI.post(remote_user, %{status: "some thing", local: false})
old_remote_post4_activity
|> Ecto.Changeset.change(%{local: false, updated_at: old_insert_date})
|> Repo.update!()
{:ok, old_repeat_activity} = CommonAPI.repeat(old_remote_post4_activity.id, local_user)
old_repeat_activity
|> Ecto.Changeset.change(%{local: true, updated_at: old_insert_date})
|> Repo.update!()
assert length(Repo.all(Object)) == 4
Mix.Tasks.Pleroma.Database.run(["prune_objects", "--keep-threads"])
assert length(Repo.all(Object)) == 4
end
test "with the --keep-threads option it keeps old threads with bookmarked posts" do
deadline = Pleroma.Config.get([:instance, :remote_post_retention_days]) + 1
old_insert_date =
Timex.now()
|> Timex.shift(days: -deadline)
|> Timex.to_naive_datetime()
|> NaiveDateTime.truncate(:second)
remote_user = insert(:user, local: false)
local_user = insert(:user, local: true)
{:ok, old_remote_post_activity} =
CommonAPI.post(remote_user, %{status: "some thing", local: false})
old_remote_post_activity
|> Ecto.Changeset.change(%{local: false, updated_at: old_insert_date})
|> Repo.update!()
Pleroma.Bookmark.create(local_user.id, old_remote_post_activity.id)
assert length(Repo.all(Object)) == 1
Mix.Tasks.Pleroma.Database.run(["prune_objects", "--keep-threads"])
assert length(Repo.all(Object)) == 1
end
end

View File

@ -0,0 +1,11 @@
defmodule Pleroma.ISO639Test do
use Pleroma.DataCase
describe "ISO639 validation" do
test "should validate a language" do
assert Pleroma.ISO639.valid_alpha2?("en")
assert Pleroma.ISO639.valid_alpha2?("ja")
refute Pleroma.ISO639.valid_alpha2?("xx")
end
end
end

View File

@ -48,6 +48,31 @@ test "wildcard domains with two subdomains" do
refute MRF.subdomain_match?(regexes, "sub.unsafe.tldanother")
end
test "wildcard on the tld" do
regexes = MRF.subdomains_regex(["somewhere.*"])
assert regexes == [~r/^(.+\.)?somewhere\.(.+)$/i]
assert MRF.subdomain_match?(regexes, "somewhere.net")
assert MRF.subdomain_match?(regexes, "somewhere.com")
assert MRF.subdomain_match?(regexes, "somewhere.somewherelese.net")
refute MRF.subdomain_match?(regexes, "somewhere")
end
test "wildcards on subdomain _and_ tld" do
regexes = MRF.subdomains_regex(["*.somewhere.*"])
assert regexes == [~r/^(.+\.)?somewhere\.(.+)$/i]
assert MRF.subdomain_match?(regexes, "somewhere.net")
assert MRF.subdomain_match?(regexes, "somewhere.com")
assert MRF.subdomain_match?(regexes, "sub.somewhere.net")
assert MRF.subdomain_match?(regexes, "sub.somewhere.com")
assert MRF.subdomain_match?(regexes, "sub.sub.somewhere.net")
assert MRF.subdomain_match?(regexes, "sub.sub.somewhere.com")
refute MRF.subdomain_match?(regexes, "somewhere")
end
test "matches are case-insensitive" do
regexes = MRF.subdomains_regex(["UnSafe.TLD", "UnSAFE2.Tld"])

View File

@ -39,6 +39,20 @@ test "a basic note validates", %{note: note} do
%{valid?: true} = ArticleNotePageValidator.cast_and_validate(note)
end
test "a note with a language validates" do
insert(:user, %{ap_id: "https://mastodon.social/users/akkoma_ap_integration_tester"})
note = File.read!("test/fixtures/mastodon/note_with_language.json") |> Jason.decode!()
%{
valid?: true,
changes: %{
contentMap: %{
"ja" => "<p>tag</p>"
}
}
} = ArticleNotePageValidator.cast_and_validate(note)
end
test "a note from factory validates" do
note = insert(:note)
%{valid?: true} = ArticleNotePageValidator.cast_and_validate(note.data)

View File

@ -12,6 +12,7 @@ defmodule Pleroma.Web.ActivityPub.ObjectValidators.BlockValidationTest do
describe "blocks" do
setup do
clear_config([:activitypub, :outgoing_blocks], true)
user = insert(:user, local: false)
blocked = insert(:user)

View File

@ -19,6 +19,12 @@ test "gets an actor for the relay" do
assert user.ap_id == "#{Pleroma.Web.Endpoint.url()}/relay"
end
test "relay actor is an application" do
# See <https://www.w3.org/TR/activitystreams-vocabulary/#dfn-application>
user = Relay.get_actor()
assert user.actor_type == "Application"
end
test "relay actor is invisible" do
user = Relay.get_actor()
assert User.invisible?(user)

View File

@ -495,8 +495,16 @@ test "removes microseconds from date (String)" do
assert Utils.to_masto_date("2015-01-23T23:50:07.123Z") == "2015-01-23T23:50:07.000Z"
end
test "returns empty string when date invalid" do
assert Utils.to_masto_date("2015-01?23T23:50:07.123Z") == ""
test "returns unix epoch when date invalid" do
assert Utils.to_masto_date("2015-01?23T23:50:07.123Z") == "1970-01-01T00:00:00Z"
end
test "returns unix epoch when date is before the introduction of the Gregorian Calendar" do
assert Utils.to_masto_date("0621-01-01T00:00:00Z") == "1970-01-01T00:00:00Z"
end
test "returns unix epoch when date is BCE" do
assert Utils.to_masto_date("-0420-01-01T00:00:00Z") == "1970-01-01T00:00:00Z"
end
end

View File

@ -73,6 +73,7 @@ test "it posts a poll" do
test "it blocks and federates", %{blocker: blocker, blocked: blocked} do
clear_config([:instance, :federating], true)
clear_config([:activitypub, :outgoing_blocks], true)
with_mock Pleroma.Web.Federator,
publish: fn _ -> nil end do

View File

@ -10,6 +10,11 @@ defmodule Pleroma.Web.MastodonAPI.FollowRequestControllerTest do
import Pleroma.Factory
defp extract_next_link_header(header) do
[_, next_link] = Regex.run(~r{<(?<next_link>.*)>; rel="next"}, header)
next_link
end
describe "locked accounts" do
setup do
user = insert(:user, is_locked: true)
@ -31,6 +36,23 @@ test "/api/v1/follow_requests works", %{user: user, conn: conn} do
assert to_string(other_user.id) == relationship["id"]
end
test "/api/v1/follow_requests paginates", %{user: user, conn: conn} do
for _ <- 1..21 do
other_user = insert(:user)
{:ok, _, _, _activity} = CommonAPI.follow(other_user, user)
{:ok, _, _} = User.follow(other_user, user, :follow_pending)
end
conn = get(conn, "/api/v1/follow_requests")
assert length(json_response_and_validate_schema(conn, 200)) == 20
assert [link_header] = get_resp_header(conn, "link")
assert link_header =~ "rel=\"next\""
next_link = extract_next_link_header(link_header)
assert next_link =~ "/api/v1/follow_requests"
conn = get(conn, next_link)
assert length(json_response_and_validate_schema(conn, 200)) == 1
end
test "/api/v1/follow_requests/:id/authorize works", %{user: user, conn: conn} do
other_user = insert(:user)

View File

@ -67,11 +67,17 @@ test "posting a status", %{conn: conn} do
|> post("/api/v1/statuses", %{
"status" => "cofe",
"spoiler_text" => "2hu",
"sensitive" => "0"
"sensitive" => "0",
"language" => "ja"
})
assert %{"content" => "cofe", "id" => id, "spoiler_text" => "2hu", "sensitive" => false} =
json_response_and_validate_schema(conn_one, 200)
assert %{
"content" => "cofe",
"id" => id,
"spoiler_text" => "2hu",
"sensitive" => false,
"language" => "ja"
} = json_response_and_validate_schema(conn_one, 200)
assert Activity.get_by_id(id)
@ -213,6 +219,18 @@ test "posting an undefined status with an attachment", %{user: user, conn: conn}
assert json_response_and_validate_schema(conn, 200)
end
test "posting a status with an invalid language", %{conn: conn} do
conn =
conn
|> put_req_header("content-type", "application/json")
|> post("/api/v1/statuses", %{
"status" => "cofe",
"language" => "invalid"
})
assert %{"error" => "Invalid language"} = json_response_and_validate_schema(conn, 422)
end
test "replying to a status", %{user: user, conn: conn} do
{:ok, replied_to} = CommonAPI.post(user, %{status: "cofe"})

View File

@ -40,11 +40,11 @@ test "GET /api/v1/pleroma/emoji/packs", %{conn: conn} do
|> get("/api/v1/pleroma/emoji/packs")
|> json_response_and_validate_schema(200)
assert resp["count"] == 4
assert resp["count"] == 5
assert resp["packs"]
|> Map.keys()
|> length() == 4
|> length() == 5
shared = resp["packs"]["test_pack"]
assert shared["files"] == %{"blank" => "blank.png", "blank2" => "blank2.png"}
@ -61,7 +61,7 @@ test "GET /api/v1/pleroma/emoji/packs", %{conn: conn} do
|> get("/api/v1/pleroma/emoji/packs?page_size=1")
|> json_response_and_validate_schema(200)
assert resp["count"] == 4
assert resp["count"] == 5
packs = Map.keys(resp["packs"])
@ -74,7 +74,7 @@ test "GET /api/v1/pleroma/emoji/packs", %{conn: conn} do
|> get("/api/v1/pleroma/emoji/packs?page_size=1&page=2")
|> json_response_and_validate_schema(200)
assert resp["count"] == 4
assert resp["count"] == 5
packs = Map.keys(resp["packs"])
assert length(packs) == 1
[pack2] = packs
@ -84,7 +84,7 @@ test "GET /api/v1/pleroma/emoji/packs", %{conn: conn} do
|> get("/api/v1/pleroma/emoji/packs?page_size=1&page=3")
|> json_response_and_validate_schema(200)
assert resp["count"] == 4
assert resp["count"] == 5
packs = Map.keys(resp["packs"])
assert length(packs) == 1
[pack3] = packs
@ -94,7 +94,7 @@ test "GET /api/v1/pleroma/emoji/packs", %{conn: conn} do
|> get("/api/v1/pleroma/emoji/packs?page_size=1&page=4")
|> json_response_and_validate_schema(200)
assert resp["count"] == 4
assert resp["count"] == 5
packs = Map.keys(resp["packs"])
assert length(packs) == 1
[pack4] = packs
@ -221,6 +221,24 @@ test "shared pack from remote and non shared from fallback-src", %{
url: "https://nonshared-pack"
} ->
text(File.read!("#{@emoji_path}/test_pack_nonshared/nonshared.zip"))
%{
method: :get,
url: "https://example.com/api/v1/pleroma/emoji/pack?name=test%20with%20spaces"
} ->
conn
|> get("/api/v1/pleroma/emoji/pack?name=test%20with%20spaces")
|> json_response_and_validate_schema(200)
|> json()
%{
method: :get,
url: "https://example.com/api/v1/pleroma/emoji/packs/archive?name=test%20with%20spaces"
} ->
conn
|> get("/api/v1/pleroma/emoji/packs/archive?name=test%20with%20spaces")
|> response(200)
|> text()
end)
assert admin_conn
@ -261,6 +279,18 @@ test "shared pack from remote and non shared from fallback-src", %{
|> json_response_and_validate_schema(200) == "ok"
refute File.exists?("#{@emoji_path}/test_pack_nonshared2")
assert admin_conn
|> put_req_header("content-type", "multipart/form-data")
|> post("/api/v1/pleroma/emoji/packs/download", %{
url: "https://example.com",
name: "test with spaces",
as: "test with spaces"
})
|> json_response_and_validate_schema(200) == "ok"
assert File.exists?("#{@emoji_path}/test with spaces/pack.json")
assert File.exists?("#{@emoji_path}/test with spaces/blank.png")
end
test "nonshareable instance", %{admin_conn: admin_conn} do

View File

@ -38,16 +38,30 @@ test "it confirms the user account", %{conn: conn, user: user} do
refute user.confirmation_token
end
test "it returns 500 if user cannot be found by id", %{conn: conn, user: user} do
conn = get(conn, "/api/account/confirm_email/0/#{user.confirmation_token}")
test "confirmation is requested twice", %{conn: conn, user: user} do
conn = get(conn, "/api/account/confirm_email/#{user.id}/#{user.confirmation_token}")
assert 302 == conn.status
assert 500 == conn.status
conn = get(conn, "/api/account/confirm_email/#{user.id}/#{user.confirmation_token}")
assert 400 == conn.status
assert "Already verified email" == conn.resp_body
user = User.get_cached_by_id(user.id)
assert user.is_confirmed
refute user.confirmation_token
end
test "it returns 500 if token is invalid", %{conn: conn, user: user} do
test "it returns 400 if user cannot be found by id", %{conn: conn, user: user} do
conn = get(conn, "/api/account/confirm_email/0/#{user.confirmation_token}")
assert 400 == conn.status
end
test "it returns 400 if token is invalid", %{conn: conn, user: user} do
conn = get(conn, "/api/account/confirm_email/#{user.id}/wrong_token")
assert 500 == conn.status
assert 400 == conn.status
end
end