From 795ea5dfc2549b50265cea2f7b7a774356a735b4 Mon Sep 17 00:00:00 2001 From: Maxim Filippov Date: Fri, 4 Oct 2019 18:58:44 +0300 Subject: [PATCH 1/9] Move HTTP verb to the header (admin_api.md) --- docs/api/admin_api.md | 190 ++++++++++++++++++++++++------------------ 1 file changed, 109 insertions(+), 81 deletions(-) diff --git a/docs/api/admin_api.md b/docs/api/admin_api.md index ee9e68cb1..045686bf4 100644 --- a/docs/api/admin_api.md +++ b/docs/api/admin_api.md @@ -2,11 +2,10 @@ Authentication is required and the user must be an admin. -## `/api/pleroma/admin/users` +## `GET /api/pleroma/admin/users` ### List users -- Method `GET` - Query Params: - *optional* `query`: **string** search term (e.g. nickname, domain, nickname@domain) - *optional* `filters`: **string** comma-separated string of filters: @@ -47,11 +46,10 @@ Authentication is required and the user must be an admin. } ``` -## `/api/pleroma/admin/users` +## `DELETE /api/pleroma/admin/users` ### Remove a user -- Method `DELETE` - Params: - `nickname` - Response: User’s nickname @@ -69,31 +67,30 @@ Authentication is required and the user must be an admin. ] - Response: User’s nickname -## `/api/pleroma/admin/users/follow` +## `POST /api/pleroma/admin/users/follow` + ### Make a user follow another user -- Methods: `POST` - Params: - - `follower`: The nickname of the follower - - `followed`: The nickname of the followed + - `follower`: The nickname of the follower + - `followed`: The nickname of the followed - Response: - - "ok" + - "ok" + +## `POST /api/pleroma/admin/users/unfollow` -## `/api/pleroma/admin/users/unfollow` ### Make a user unfollow another user -- Methods: `POST` - Params: - - `follower`: The nickname of the follower - - `followed`: The nickname of the followed + - `follower`: The nickname of the follower + - `followed`: The nickname of the followed - Response: - - "ok" + - "ok" -## `/api/pleroma/admin/users/:nickname/toggle_activation` +## `PATCH /api/pleroma/admin/users/:nickname/toggle_activation` ### Toggle user activation -- Method: `PATCH` - Params: - `nickname` - Response: User’s object @@ -106,27 +103,26 @@ Authentication is required and the user must be an admin. } ``` -## `/api/pleroma/admin/users/tag` +## `PUT /api/pleroma/admin/users/tag` ### Tag a list of users -- Method: `PUT` - Params: - `nicknames` (array) - `tags` (array) +## `DELETE /api/pleroma/admin/users/tag` + ### Untag a list of users -- Method: `DELETE` - Params: - `nicknames` (array) - `tags` (array) -## `/api/pleroma/admin/users/:nickname/permission_group` +## `GET /api/pleroma/admin/users/:nickname/permission_group` ### Get user user permission groups membership -- Method: `GET` - Params: none - Response: @@ -137,13 +133,12 @@ Authentication is required and the user must be an admin. } ``` -## `/api/pleroma/admin/users/:nickname/permission_group/:permission_group` +## `GET /api/pleroma/admin/users/:nickname/permission_group/:permission_group` Note: Available `:permission_group` is currently moderator and admin. 404 is returned when the permission group doesn’t exist. ### Get user user permission groups membership per permission group -- Method: `GET` - Params: none - Response: @@ -154,48 +149,47 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret } ``` +## `POST /api/pleroma/admin/users/:nickname/permission_group/:permission_group` + ### Add user in permission group -- Method: `POST` - Params: none - Response: - On failure: `{"error": "…"}` - On success: JSON of the `user.info` +## `DELETE /api/pleroma/admin/users/:nickname/permission_group/:permission_group` + ### Remove user from permission group -- Method: `DELETE` - Params: none - Response: - On failure: `{"error": "…"}` - On success: JSON of the `user.info` - Note: An admin cannot revoke their own admin status. -## `/api/pleroma/admin/users/:nickname/activation_status` +## `PUT /api/pleroma/admin/users/:nickname/activation_status` ### Active or deactivate a user -- Method: `PUT` - Params: - `nickname` - `status` BOOLEAN field, false value means deactivation. -## `/api/pleroma/admin/users/:nickname_or_id` +## `GET /api/pleroma/admin/users/:nickname_or_id` ### Retrive the details of a user -- Method: `GET` - Params: - `nickname` or `id` - Response: - On failure: `Not found` - On success: JSON of the user -## `/api/pleroma/admin/users/:nickname_or_id/statuses` +## `GET /api/pleroma/admin/users/:nickname_or_id/statuses` ### Retrive user's latest statuses -- Method: `GET` - Params: - `nickname` or `id` - *optional* `page_size`: number of statuses to return (default is `20`) @@ -204,29 +198,28 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret - On failure: `Not found` - On success: JSON array of user's latest statuses -## `/api/pleroma/admin/relay` +## `POST /api/pleroma/admin/relay` ### Follow a Relay -- Methods: `POST` - Params: - `relay_url` - Response: - On success: URL of the followed relay +## `DELETE /api/pleroma/admin/relay` + ### Unfollow a Relay -- Methods: `DELETE` - Params: - `relay_url` - Response: - On success: URL of the unfollowed relay -## `/api/pleroma/admin/users/invite_token` +## `POST /api/pleroma/admin/users/invite_token` ### Create an account registration invite token -- Methods: `POST` - Params: - *optional* `max_use` (integer) - *optional* `expires_at` (date string e.g. "2019-04-07") @@ -244,11 +237,10 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret } ``` -## `/api/pleroma/admin/users/invites` +## `GET /api/pleroma/admin/users/invites` ### Get a list of generated invites -- Methods: `GET` - Params: none - Response: @@ -270,11 +262,10 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret } ``` -## `/api/pleroma/admin/users/revoke_invite` +## `POST /api/pleroma/admin/users/revoke_invite` ### Revoke invite by token -- Methods: `POST` - Params: - `token` - Response: @@ -292,21 +283,18 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret } ``` - -## `/api/pleroma/admin/users/email_invite` +## `POST /api/pleroma/admin/users/email_invite` ### Sends registration invite via email -- Methods: `POST` - Params: - `email` - `name`, optional -## `/api/pleroma/admin/users/:nickname/password_reset` +## `GET /api/pleroma/admin/users/:nickname/password_reset` ### Get a password reset token for a given nickname -- Methods: `GET` - Params: none - Response: @@ -317,18 +305,17 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret } ``` - -## `/api/pleroma/admin/users/:nickname/force_password_reset` +## `PATCH /api/pleroma/admin/users/:nickname/force_password_reset` ### Force passord reset for a user with a given nickname -- Methods: `PATCH` - Params: none - Response: none (code `204`) -## `/api/pleroma/admin/reports` +## `GET /api/pleroma/admin/reports` + ### Get a list of reports -- Method `GET` + - Params: - *optional* `state`: **string** the state of reports. Valid values are `open`, `closed` and `resolved` - *optional* `limit`: **integer** the number of records to retrieve @@ -343,7 +330,7 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret ```json { - "total" : 1, + "totalReports" : 1, "reports": [ { "account": { @@ -481,13 +468,24 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret } ] } + ], + "totalGroupedReports": 1, + "groupedReports": [ + { + "date": "2019-01-01", // date of the latest report + "account": { ... }, // author of the reported status + "status": { ... }, // reported status + "actors": [{ ... }, { ... }], // accounts that sent reports on the status + "reports": [{ ... }] + } ] } ``` -## `/api/pleroma/admin/reports/:id` +## `GET /api/pleroma/admin/reports/:id` + ### Get an individual report -- Method `GET` + - Params: - `id` - Response: @@ -496,22 +494,41 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret - 404 Not Found `"Not found"` - On success: JSON, Report object (see above) -## `/api/pleroma/admin/reports/:id` -### Change the state of the report -- Method `PUT` +## `PATCH /api/pleroma/admin/reports` + +### Change the state of one or multiple reports + - Params: - - `id` - - `state`: required, the new state. Valid values are `open`, `closed` and `resolved` + +```json + `reports`: [ + { + `id`, // required, report id + `state` // required, the new state. Valid values are `open`, `closed` and `resolved` + }, + ... + ] +``` + - Response: - On failure: - - 400 Bad Request `"Unsupported state"` - - 403 Forbidden `{"error": "error_msg"}` - - 404 Not Found `"Not found"` - - On success: JSON, Report object (see above) + - 400 Bad Request, JSON: + + ```json + [ + { + `id`, // report id + `error` // error message + } + ] + ``` + + - On success: `204`, empty response + +## `POST /api/pleroma/admin/reports/:id/respond` -## `/api/pleroma/admin/reports/:id/respond` ### Respond to a report -- Method `POST` + - Params: - `id` - `status`: required, the message @@ -581,9 +598,10 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret } ``` -## `/api/pleroma/admin/statuses/:id` +## `PUT /api/pleroma/admin/statuses/:id` + ### Change the scope of an individual reported status -- Method `PUT` + - Params: - `id` - `sensitive`: optional, valid values are `true` or `false` @@ -595,9 +613,10 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret - 404 Not Found `"Not found"` - On success: JSON, Mastodon Status entity -## `/api/pleroma/admin/statuses/:id` +## `DELETE /api/pleroma/admin/statuses/:id` + ### Delete an individual reported status -- Method `DELETE` + - Params: - `id` - Response: @@ -606,11 +625,12 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret - 404 Not Found `"Not found"` - On success: 200 OK `{}` +## `GET /api/pleroma/admin/config/migrate_to_db` -## `/api/pleroma/admin/config/migrate_to_db` ### Run mix task pleroma.config migrate_to_db + Copy settings on key `:pleroma` to DB. -- Method `GET` + - Params: none - Response: @@ -618,9 +638,12 @@ Copy settings on key `:pleroma` to DB. {} ``` -## `/api/pleroma/admin/config/migrate_from_db` +## `GET /api/pleroma/admin/config/migrate_from_db` + ### Run mix task pleroma.config migrate_from_db + Copy all settings from DB to `config/prod.exported_from_db.secret.exs` with deletion from DB. + - Method `GET` - Params: none - Response: @@ -629,10 +652,12 @@ Copy all settings from DB to `config/prod.exported_from_db.secret.exs` with dele {} ``` -## `/api/pleroma/admin/config` +## `GET /api/pleroma/admin/config` + ### List config settings + List config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`. -- Method `GET` + - Params: none - Response: @@ -648,8 +673,10 @@ List config settings only works with `:pleroma => :instance => :dynamic_configur } ``` -## `/api/pleroma/admin/config` +## `POST /api/pleroma/admin/config` + ### Update config settings + Updating config settings only works with `:pleroma => :instance => :dynamic_configuration` setting to `true`. Module name can be passed as string, which starts with `Pleroma`, e.g. `"Pleroma.Upload"`. Atom keys and values can be passed with `:` in the beginning, e.g. `":upload"`. @@ -672,7 +699,6 @@ Compile time settings (need instance reboot): - `Pleroma.Upload` -> `:proxy_remote` - `:instance` -> `:upload_limit` -- Method `POST` - Params: - `configs` => [ - `group` (string) @@ -727,9 +753,10 @@ Compile time settings (need instance reboot): } ``` -## `/api/pleroma/admin/moderation_log` +## `GET /api/pleroma/admin/moderation_log` + ### Get moderation log -- Method `GET` + - Params: - *optional* `page`: **integer** page number - *optional* `page_size`: **integer** number of log entries per page (default is `50`) @@ -756,8 +783,9 @@ Compile time settings (need instance reboot): ``` ## `POST /api/pleroma/admin/reload_emoji` + ### Reload the instance's custom emoji -* Method `POST` -* Authentication: required -* Params: None -* Response: JSON, "ok" and 200 status + +- Authentication: required +- Params: None +- Response: JSON, "ok" and 200 status From 8dcc2f9f5ecbbc81bc026c85582695de4fbc1a0f Mon Sep 17 00:00:00 2001 From: Maxim Filippov Date: Fri, 4 Oct 2019 19:00:58 +0300 Subject: [PATCH 2/9] Admin API: Allow changing the state of multiple reports at once --- CHANGELOG.md | 1 + lib/pleroma/web/activity_pub/utils.ex | 12 +++ .../web/admin_api/admin_api_controller.ex | 29 +++--- lib/pleroma/web/common_api/common_api.ex | 7 ++ lib/pleroma/web/router.ex | 2 +- .../admin_api/admin_api_controller_test.exs | 89 +++++++++++++++---- test/web/common_api/common_api_test.exs | 29 ++++++ 7 files changed, 142 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a71a9dae6..d7afed783 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). ### Changed - **Breaking:** Elixir >=1.8 is now required (was >= 1.7) - **Breaking:** Admin API: Return link alongside with token on password reset +- **Breaking:** Admin API: Changing report state now uses `PATCH` (it was `PUT` before) and allows updating multiple reports at once (API changed) - Replaced [pleroma_job_queue](https://git.pleroma.social/pleroma/pleroma_job_queue) and `Pleroma.Web.Federator.RetryQueue` with [Oban](https://github.com/sorentwo/oban) (see [`docs/config.md`](docs/config.md) on migrating customized worker / retry settings) - Introduced [quantum](https://github.com/quantum-elixir/quantum-core) job scheduler - Admin API: Return `total` when querying for reports diff --git a/lib/pleroma/web/activity_pub/utils.ex b/lib/pleroma/web/activity_pub/utils.ex index 0828591ee..824957314 100644 --- a/lib/pleroma/web/activity_pub/utils.ex +++ b/lib/pleroma/web/activity_pub/utils.ex @@ -672,6 +672,18 @@ def update_report_state(%Activity{} = activity, state) when state in @supported_ |> Repo.update() end + def update_report_state(activity_ids, state) when state in @supported_report_states do + activities_num = length(activity_ids) + + from(a in Activity, where: a.id in ^activity_ids) + |> update(set: [data: fragment("jsonb_set(data, '{state}', ?)", ^state)]) + |> Repo.update_all([]) + |> case do + {^activities_num, _} -> :ok + _ -> {:error, activity_ids} + end + end + def update_report_state(_, _), do: {:error, "Unsupported state"} def update_activity_visibility(activity, visibility) when visibility in @valid_visibilities do diff --git a/lib/pleroma/web/admin_api/admin_api_controller.ex b/lib/pleroma/web/admin_api/admin_api_controller.ex index 21da8a7ff..0e8c9dac8 100644 --- a/lib/pleroma/web/admin_api/admin_api_controller.ex +++ b/lib/pleroma/web/admin_api/admin_api_controller.ex @@ -480,17 +480,26 @@ def report_show(conn, %{"id" => id}) do end end - def report_update_state(%{assigns: %{user: admin}} = conn, %{"id" => id, "state" => state}) do - with {:ok, report} <- CommonAPI.update_report_state(id, state) do - ModerationLog.insert_log(%{ - action: "report_update", - actor: admin, - subject: report - }) + def reports_update(%{assigns: %{user: admin}} = conn, %{"reports" => reports}) do + result = + reports + |> Enum.map(fn report -> + with {:ok, activity} <- CommonAPI.update_report_state(report["id"], report["state"]) do + ModerationLog.insert_log(%{ + action: "report_update", + actor: admin, + subject: activity + }) - conn - |> put_view(ReportView) - |> render("show.json", Report.extract_report_info(report)) + activity + else + {:error, message} -> %{id: report["id"], error: message} + end + end) + + case Enum.any?(result, &Map.has_key?(&1, :error)) do + true -> json_response(conn, :bad_request, result) + false -> json_response(conn, :no_content, "") end end diff --git a/lib/pleroma/web/common_api/common_api.ex b/lib/pleroma/web/common_api/common_api.ex index ce73b3270..2b80598ea 100644 --- a/lib/pleroma/web/common_api/common_api.ex +++ b/lib/pleroma/web/common_api/common_api.ex @@ -346,6 +346,13 @@ defp get_reported_account(account_id) do end end + def update_report_state(activity_ids, state) when is_list(activity_ids) do + case Utils.update_report_state(activity_ids, state) do + :ok -> {:ok, activity_ids} + _ -> {:error, dgettext("errors", "Could not update state")} + end + end + def update_report_state(activity_id, state) do with %Activity{} = activity <- Activity.get_by_id(activity_id) do Utils.update_report_state(activity, state) diff --git a/lib/pleroma/web/router.ex b/lib/pleroma/web/router.ex index f91af8137..563b01dc5 100644 --- a/lib/pleroma/web/router.ex +++ b/lib/pleroma/web/router.ex @@ -194,7 +194,7 @@ defmodule Pleroma.Web.Router do get("/reports", AdminAPIController, :list_reports) get("/reports/:id", AdminAPIController, :report_show) - put("/reports/:id", AdminAPIController, :report_update_state) + patch("/reports", AdminAPIController, :reports_update) post("/reports/:id/respond", AdminAPIController, :report_respond) put("/statuses/:id", AdminAPIController, :status_update) diff --git a/test/web/admin_api/admin_api_controller_test.exs b/test/web/admin_api/admin_api_controller_test.exs index b5c355e66..cec3570eb 100644 --- a/test/web/admin_api/admin_api_controller_test.exs +++ b/test/web/admin_api/admin_api_controller_test.exs @@ -1224,7 +1224,7 @@ test "returns 404 when report id is invalid", %{conn: conn} do end end - describe "PUT /api/pleroma/admin/reports/:id" do + describe "PATCH /api/pleroma/admin/reports" do setup %{conn: conn} do admin = insert(:user, info: %{is_admin: true}) [reporter, target_user] = insert_pair(:user) @@ -1237,16 +1237,32 @@ test "returns 404 when report id is invalid", %{conn: conn} do "status_ids" => [activity.id] }) - %{conn: assign(conn, :user, admin), id: report_id, admin: admin} + {:ok, %{id: second_report_id}} = + CommonAPI.report(reporter, %{ + "account_id" => target_user.id, + "comment" => "I feel very offended", + "status_ids" => [activity.id] + }) + + %{ + conn: assign(conn, :user, admin), + id: report_id, + admin: admin, + second_report_id: second_report_id + } end test "mark report as resolved", %{conn: conn, id: id, admin: admin} do - response = - conn - |> put("/api/pleroma/admin/reports/#{id}", %{"state" => "resolved"}) - |> json_response(:ok) + conn + |> patch("/api/pleroma/admin/reports", %{ + "reports" => [ + %{"state" => "resolved", "id" => id} + ] + }) + |> json_response(:no_content) - assert response["state"] == "resolved" + activity = Activity.get_by_id(id) + assert activity.data["state"] == "resolved" log_entry = Repo.one(ModerationLog) @@ -1255,12 +1271,16 @@ test "mark report as resolved", %{conn: conn, id: id, admin: admin} do end test "closes report", %{conn: conn, id: id, admin: admin} do - response = - conn - |> put("/api/pleroma/admin/reports/#{id}", %{"state" => "closed"}) - |> json_response(:ok) + conn + |> patch("/api/pleroma/admin/reports", %{ + "reports" => [ + %{"state" => "closed", "id" => id} + ] + }) + |> json_response(:no_content) - assert response["state"] == "closed" + activity = Activity.get_by_id(id) + assert activity.data["state"] == "closed" log_entry = Repo.one(ModerationLog) @@ -1271,17 +1291,54 @@ test "closes report", %{conn: conn, id: id, admin: admin} do test "returns 400 when state is unknown", %{conn: conn, id: id} do conn = conn - |> put("/api/pleroma/admin/reports/#{id}", %{"state" => "test"}) + |> patch("/api/pleroma/admin/reports", %{ + "reports" => [ + %{"state" => "test", "id" => id} + ] + }) - assert json_response(conn, :bad_request) == "Unsupported state" + assert hd(json_response(conn, :bad_request))["error"] == "Unsupported state" end test "returns 404 when report is not exist", %{conn: conn} do conn = conn - |> put("/api/pleroma/admin/reports/test", %{"state" => "closed"}) + |> patch("/api/pleroma/admin/reports", %{ + "reports" => [ + %{"state" => "closed", "id" => "test"} + ] + }) - assert json_response(conn, :not_found) == "Not found" + assert hd(json_response(conn, :bad_request))["error"] == "not_found" + end + + test "updates state of multiple reports", %{ + conn: conn, + id: id, + admin: admin, + second_report_id: second_report_id + } do + conn + |> patch("/api/pleroma/admin/reports", %{ + "reports" => [ + %{"state" => "resolved", "id" => id}, + %{"state" => "closed", "id" => second_report_id} + ] + }) + |> json_response(:no_content) + + activity = Activity.get_by_id(id) + second_activity = Activity.get_by_id(second_report_id) + assert activity.data["state"] == "resolved" + assert second_activity.data["state"] == "closed" + + [first_log_entry, second_log_entry] = Repo.all(ModerationLog) + + assert ModerationLog.get_log_entry_message(first_log_entry) == + "@#{admin.nickname} updated report ##{id} with 'resolved' state" + + assert ModerationLog.get_log_entry_message(second_log_entry) == + "@#{admin.nickname} updated report ##{second_report_id} with 'closed' state" end end diff --git a/test/web/common_api/common_api_test.exs b/test/web/common_api/common_api_test.exs index 2d3c41e82..c57fdb6af 100644 --- a/test/web/common_api/common_api_test.exs +++ b/test/web/common_api/common_api_test.exs @@ -423,6 +423,35 @@ test "does not update report state when state is unsupported" do assert CommonAPI.update_report_state(report_id, "test") == {:error, "Unsupported state"} end + + test "updates state of multiple reports" do + [reporter, target_user] = insert_pair(:user) + activity = insert(:note_activity, user: target_user) + + {:ok, %Activity{id: first_report_id}} = + CommonAPI.report(reporter, %{ + "account_id" => target_user.id, + "comment" => "I feel offended", + "status_ids" => [activity.id] + }) + + {:ok, %Activity{id: second_report_id}} = + CommonAPI.report(reporter, %{ + "account_id" => target_user.id, + "comment" => "I feel very offended!", + "status_ids" => [activity.id] + }) + + {:ok, report_ids} = + CommonAPI.update_report_state([first_report_id, second_report_id], "resolved") + + first_report = Activity.get_by_id(first_report_id) + second_report = Activity.get_by_id(second_report_id) + + assert report_ids -- [first_report_id, second_report_id] == [] + assert first_report.data["state"] == "resolved" + assert second_report.data["state"] == "resolved" + end end describe "reblog muting" do From 7aceaa517be7b109a9acc15fb4914535b536b66c Mon Sep 17 00:00:00 2001 From: Maxim Filippov Date: Mon, 7 Oct 2019 15:01:18 +0300 Subject: [PATCH 3/9] Admin API: Reports, grouped by status --- CHANGELOG.md | 1 + docs/api/admin_api.md | 30 +++++-- lib/pleroma/activity.ex | 21 +++++ lib/pleroma/web/activity_pub/utils.ex | 89 +++++++++++++++++++ .../web/admin_api/admin_api_controller.ex | 19 ++-- .../web/admin_api/views/report_view.ex | 20 +++++ lib/pleroma/web/router.ex | 1 + .../admin_api/admin_api_controller_test.exs | 69 +++++++++++++- 8 files changed, 230 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d7afed783..7956a6527 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Pleroma API: `GET /api/v1/pleroma/accounts/:id/scrobbles` to get a list of recently scrobbled items - Pleroma API: `POST /api/v1/pleroma/scrobble` to scrobble a media item - Mastodon API: Add `upload_limit`, `avatar_upload_limit`, `background_upload_limit`, and `banner_upload_limit` to `/api/v1/instance` +- Admin API: Add ability to fetch reports, grouped by status `GET /api/pleroma/admin/grouped_reports` ### Changed - **Breaking:** Elixir >=1.8 is now required (was >= 1.7) diff --git a/docs/api/admin_api.md b/docs/api/admin_api.md index 045686bf4..e8232225c 100644 --- a/docs/api/admin_api.md +++ b/docs/api/admin_api.md @@ -468,18 +468,32 @@ Note: Available `:permission_group` is currently moderator and admin. 404 is ret } ] } - ], - "totalGroupedReports": 1, - "groupedReports": [ + ] +} +``` + +## `GET /api/pleroma/admin/grouped_reports` + +### Get a list of reports, grouped by status + +- Params: none +- On success: JSON, returns a list of reports, where: + - `date`: date of the latest report + - `account`: the user who has been reported (see `/api/pleroma/admin/reports` for reference) + - `status`: reported status (see `/api/pleroma/admin/reports` for reference) + - `actors`: users who had reported this status (see `/api/pleroma/admin/reports` for reference) + - `reports`: reports (see `/api/pleroma/admin/reports` for reference) + +```json + "reports": [ { - "date": "2019-01-01", // date of the latest report - "account": { ... }, // author of the reported status - "status": { ... }, // reported status - "actors": [{ ... }, { ... }], // accounts that sent reports on the status + "date": "2019-10-07T12:31:39.615149Z", + "account": { ... }, + "status": { ... }, + "actors": [{ ... }, { ... }], "reports": [{ ... }] } ] -} ``` ## `GET /api/pleroma/admin/reports/:id` diff --git a/lib/pleroma/activity.ex b/lib/pleroma/activity.ex index c1065611b..daf0ed89f 100644 --- a/lib/pleroma/activity.ex +++ b/lib/pleroma/activity.ex @@ -41,6 +41,9 @@ defmodule Pleroma.Activity do field(:actor, :string) field(:recipients, {:array, :string}, default: []) field(:thread_muted?, :boolean, virtual: true) + + # This is a fake relation, do not use outside of with_preloaded_user_actor/with_joined_user_actor + has_one(:user_actor, User, on_delete: :nothing, foreign_key: :id) # This is a fake relation, do not use outside of with_preloaded_bookmark/get_bookmark has_one(:bookmark, Bookmark) has_many(:notifications, Notification, on_delete: :delete_all) @@ -86,6 +89,24 @@ def with_preloaded_object(query, join_type \\ :inner) do |> preload([activity, object: object], object: object) end + def with_joined_user_actor(query, join_type \\ :inner) do + join(query, join_type, [activity], u in User, + on: + fragment( + "? = ?->>'actor'", + u.ap_id, + activity.data + ), + as: :user_actor + ) + end + + def with_preloaded_user_actor(query, join_type \\ :inner) do + query + |> with_joined_user_actor(join_type) + |> preload([activity, user_actor: user_actor], user_actor: user_actor) + end + def with_preloaded_bookmark(query, %User{} = user) do from([a] in query, left_join: b in Bookmark, diff --git a/lib/pleroma/web/activity_pub/utils.ex b/lib/pleroma/web/activity_pub/utils.ex index 824957314..74eb994ab 100644 --- a/lib/pleroma/web/activity_pub/utils.ex +++ b/lib/pleroma/web/activity_pub/utils.ex @@ -6,11 +6,13 @@ defmodule Pleroma.Web.ActivityPub.Utils do alias Ecto.Changeset alias Ecto.UUID alias Pleroma.Activity + alias Pleroma.Activity.Queries alias Pleroma.Notification alias Pleroma.Object alias Pleroma.Repo alias Pleroma.User alias Pleroma.Web + alias Pleroma.Web.ActivityPub.ActivityPub alias Pleroma.Web.ActivityPub.Visibility alias Pleroma.Web.Endpoint alias Pleroma.Web.Router.Helpers @@ -664,6 +666,93 @@ def fetch_ordered_collection(from, pages_left, acc \\ []) do #### Report-related helpers + def get_reports(params, page, page_size) do + params = + params + |> Map.put("type", "Flag") + |> Map.put("skip_preload", true) + |> Map.put("total", true) + |> Map.put("limit", page_size) + |> Map.put("offset", (page - 1) * page_size) + + ActivityPub.fetch_activities([], params, :offset) + end + + @spec get_reports_grouped_by_status() :: %{ + required(:groups) => [ + %{ + required(:date) => String.t(), + required(:account) => %User{}, + required(:status) => %Activity{}, + required(:actors) => [%User{}], + required(:reports) => [%Activity{}] + } + ], + required(:total) => integer + } + def get_reports_grouped_by_status do + paginated_activities = get_reported_status_ids() + + groups = + paginated_activities + |> Enum.map(fn entry -> + status = + Activity + |> Queries.by_ap_id(entry[:activity_id]) + |> Activity.with_preloaded_object(:left) + |> Activity.with_preloaded_user_actor() + |> Repo.one() + + reports = get_reports_by_status_id(status.data["id"]) + + max_date = + Enum.max_by(reports, &Pleroma.Web.CommonAPI.Utils.to_masto_date(&1.data["published"])).data[ + "published" + ] + + actors = Enum.map(reports, & &1.user_actor) + + %{ + date: max_date, + account: status.user_actor, + status: status, + actors: actors, + reports: reports + } + end) + + %{ + groups: groups + } + end + + def get_reports_by_status_id(status_id) do + from(a in Activity, + where: fragment("(?)->>'type' = 'Flag'", a.data), + where: fragment("(?)->'object' \\? (?)", a.data, ^status_id) + ) + |> Activity.with_preloaded_user_actor() + |> Repo.all() + end + + @spec get_reported_status_ids() :: %{ + required(:items) => [%Activity{}], + required(:total) => integer + } + def get_reported_status_ids do + from(a in Activity, + where: fragment("(?)->>'type' = 'Flag'", a.data), + select: %{ + date: fragment("max(?->>'published') date", a.data), + activity_id: + fragment("jsonb_array_elements_text((? #- '{object,0}')->'object') activity_id", a.data) + }, + group_by: fragment("activity_id"), + order_by: fragment("date DESC") + ) + |> Repo.all() + end + def update_report_state(%Activity{} = activity, state) when state in @supported_report_states do new_data = Map.put(activity.data, "state", state) diff --git a/lib/pleroma/web/admin_api/admin_api_controller.ex b/lib/pleroma/web/admin_api/admin_api_controller.ex index 0e8c9dac8..463dd327a 100644 --- a/lib/pleroma/web/admin_api/admin_api_controller.ex +++ b/lib/pleroma/web/admin_api/admin_api_controller.ex @@ -10,6 +10,7 @@ defmodule Pleroma.Web.AdminAPI.AdminAPIController do alias Pleroma.UserInviteToken alias Pleroma.Web.ActivityPub.ActivityPub alias Pleroma.Web.ActivityPub.Relay + alias Pleroma.Web.ActivityPub.Utils alias Pleroma.Web.AdminAPI.AccountView alias Pleroma.Web.AdminAPI.Config alias Pleroma.Web.AdminAPI.ConfigView @@ -455,19 +456,15 @@ def force_password_reset(conn, %{"nickname" => nickname}) do def list_reports(conn, params) do {page, page_size} = page_params(params) - params = - params - |> Map.put("type", "Flag") - |> Map.put("skip_preload", true) - |> Map.put("total", true) - |> Map.put("limit", page_size) - |> Map.put("offset", (page - 1) * page_size) - - reports = ActivityPub.fetch_activities([], params, :offset) - conn |> put_view(ReportView) - |> render("index.json", %{reports: reports}) + |> render("index.json", %{reports: Utils.get_reports(params, page, page_size)}) + end + + def list_grouped_reports(conn, _params) do + conn + |> put_view(ReportView) + |> render("index_grouped.json", Utils.get_reports_grouped_by_status()) end def report_show(conn, %{"id" => id}) do diff --git a/lib/pleroma/web/admin_api/views/report_view.ex b/lib/pleroma/web/admin_api/views/report_view.ex index 101a74c63..ac25925da 100644 --- a/lib/pleroma/web/admin_api/views/report_view.ex +++ b/lib/pleroma/web/admin_api/views/report_view.ex @@ -42,6 +42,26 @@ def render("show.json", %{report: report, user: user, account: account, statuses } end + def render("index_grouped.json", %{groups: groups}) do + reports = + Enum.map(groups, fn group -> + %{ + date: group[:date], + account: merge_account_views(group[:account]), + status: StatusView.render("show.json", %{activity: group[:status]}), + actors: Enum.map(group[:actors], &merge_account_views/1), + reports: + group[:reports] + |> Enum.map(&Report.extract_report_info(&1)) + |> Enum.map(&render(__MODULE__, "show.json", &1)) + } + end) + + %{ + reports: reports + } + end + defp merge_account_views(%User{} = user) do Pleroma.Web.MastodonAPI.AccountView.render("show.json", %{user: user}) |> Map.merge(Pleroma.Web.AdminAPI.AccountView.render("show.json", %{user: user})) diff --git a/lib/pleroma/web/router.ex b/lib/pleroma/web/router.ex index 563b01dc5..b895a7b7e 100644 --- a/lib/pleroma/web/router.ex +++ b/lib/pleroma/web/router.ex @@ -193,6 +193,7 @@ defmodule Pleroma.Web.Router do get("/users/:nickname/statuses", AdminAPIController, :list_user_statuses) get("/reports", AdminAPIController, :list_reports) + get("/grouped_reports", AdminAPIController, :list_grouped_reports) get("/reports/:id", AdminAPIController, :report_show) patch("/reports", AdminAPIController, :reports_update) post("/reports/:id/respond", AdminAPIController, :report_respond) diff --git a/test/web/admin_api/admin_api_controller_test.exs b/test/web/admin_api/admin_api_controller_test.exs index cec3570eb..daa0631db 100644 --- a/test/web/admin_api/admin_api_controller_test.exs +++ b/test/web/admin_api/admin_api_controller_test.exs @@ -1461,7 +1461,74 @@ test "returns 403 when requested by anonymous" do end end - # + describe "GET /api/pleroma/admin/grouped_reports" do + setup %{conn: conn} do + admin = insert(:user, info: %{is_admin: true}) + [reporter, target_user] = insert_pair(:user) + + date1 = (DateTime.to_unix(DateTime.utc_now()) + 1000) |> DateTime.from_unix!() + date2 = (DateTime.to_unix(DateTime.utc_now()) + 2000) |> DateTime.from_unix!() + date3 = (DateTime.to_unix(DateTime.utc_now()) + 3000) |> DateTime.from_unix!() + + first_status = + insert(:note_activity, user: target_user, data_attrs: %{"published" => date1}) + + second_status = + insert(:note_activity, user: target_user, data_attrs: %{"published" => date2}) + + third_status = + insert(:note_activity, user: target_user, data_attrs: %{"published" => date3}) + + %{ + conn: assign(conn, :user, admin), + reporter: reporter, + target_user: target_user, + first_status: first_status, + second_status: second_status, + third_status: third_status + } + end + + test "returns reports grouped by status", %{ + conn: conn, + reporter: reporter, + target_user: target_user, + first_status: first_status, + second_status: second_status, + third_status: third_status + } do + {:ok, %{id: _}} = + CommonAPI.report(reporter, %{ + "account_id" => target_user.id, + "status_ids" => [first_status.id, second_status.id, third_status.id] + }) + + {:ok, %{id: _}} = + CommonAPI.report(reporter, %{ + "account_id" => target_user.id, + "status_ids" => [first_status.id, second_status.id] + }) + + {:ok, %{id: _}} = + CommonAPI.report(reporter, %{ + "account_id" => target_user.id, + "status_ids" => [first_status.id] + }) + + response = + conn + |> get("/api/pleroma/admin/grouped_reports") + |> json_response(:ok) + + assert length(response["reports"]) == 3 + [third_group, second_group, first_group] = response["reports"] + + assert length(third_group["reports"]) == 3 + assert length(second_group["reports"]) == 2 + assert length(first_group["reports"]) == 1 + end + end + describe "POST /api/pleroma/admin/reports/:id/respond" do setup %{conn: conn} do admin = insert(:user, info: %{is_admin: true}) From aa7fd616c7cfeb84551af2170886856a815dc498 Mon Sep 17 00:00:00 2001 From: Maxim Filippov Date: Mon, 7 Oct 2019 16:03:23 +0300 Subject: [PATCH 4/9] Line is too long! --- lib/pleroma/activity.ex | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/pleroma/activity.ex b/lib/pleroma/activity.ex index daf0ed89f..7b77f72c2 100644 --- a/lib/pleroma/activity.ex +++ b/lib/pleroma/activity.ex @@ -42,7 +42,8 @@ defmodule Pleroma.Activity do field(:recipients, {:array, :string}, default: []) field(:thread_muted?, :boolean, virtual: true) - # This is a fake relation, do not use outside of with_preloaded_user_actor/with_joined_user_actor + # This is a fake relation, + # do not use outside of with_preloaded_user_actor/with_joined_user_actor has_one(:user_actor, User, on_delete: :nothing, foreign_key: :id) # This is a fake relation, do not use outside of with_preloaded_bookmark/get_bookmark has_one(:bookmark, Bookmark) From f171095960d172d54015b28e8da302b5745dca86 Mon Sep 17 00:00:00 2001 From: Maxim Filippov Date: Wed, 6 Nov 2019 21:25:46 +1000 Subject: [PATCH 5/9] Grouped reports with status data baked in --- lib/pleroma/web/activity_pub/utils.ex | 58 ++++----- .../web/admin_api/views/report_view.ex | 4 +- .../admin_api/admin_api_controller_test.exs | 121 ++++++++++++++---- 3 files changed, 123 insertions(+), 60 deletions(-) diff --git a/lib/pleroma/web/activity_pub/utils.ex b/lib/pleroma/web/activity_pub/utils.ex index 57349e304..5a51b7884 100644 --- a/lib/pleroma/web/activity_pub/utils.ex +++ b/lib/pleroma/web/activity_pub/utils.ex @@ -6,7 +6,6 @@ defmodule Pleroma.Web.ActivityPub.Utils do alias Ecto.Changeset alias Ecto.UUID alias Pleroma.Activity - alias Pleroma.Activity.Queries alias Pleroma.Notification alias Pleroma.Object alias Pleroma.Repo @@ -697,8 +696,8 @@ def get_reports(params, page, page_size) do required(:groups) => [ %{ required(:date) => String.t(), - required(:account) => %User{}, - required(:status) => %Activity{}, + required(:account) => %{}, + required(:status) => %{}, required(:actors) => [%User{}], required(:reports) => [%Activity{}] } @@ -706,32 +705,23 @@ def get_reports(params, page, page_size) do required(:total) => integer } def get_reports_grouped_by_status do - paginated_activities = get_reported_status_ids() - groups = - paginated_activities + get_reported_status_ids() |> Enum.map(fn entry -> - status = - Activity - |> Queries.by_ap_id(entry[:activity_id]) - |> Activity.with_preloaded_object(:left) - |> Activity.with_preloaded_user_actor() - |> Repo.one() - - reports = get_reports_by_status_id(status.data["id"]) - - max_date = - Enum.max_by(reports, &Pleroma.Web.CommonAPI.Utils.to_masto_date(&1.data["published"])).data[ - "published" - ] - + activity = Jason.decode!(entry.activity) + reports = get_reports_by_status_id(activity["id"]) + max_date = Enum.max_by(reports, &NaiveDateTime.from_iso8601!(&1.data["published"])) actors = Enum.map(reports, & &1.user_actor) %{ - date: max_date, - account: status.user_actor, - status: status, - actors: actors, + date: max_date.data["published"], + account: activity["actor"], + status: %{ + id: activity["id"], + content: activity["content"], + published: activity["published"] + }, + actors: Enum.uniq(actors), reports: reports } end) @@ -741,28 +731,30 @@ def get_reports_grouped_by_status do } end - def get_reports_by_status_id(status_id) do + def get_reports_by_status_id(ap_id) do from(a in Activity, where: fragment("(?)->>'type' = 'Flag'", a.data), - where: fragment("(?)->'object' \\? (?)", a.data, ^status_id) + where: fragment("(?)->'object' @> ?", a.data, ^[%{id: ap_id}]) ) |> Activity.with_preloaded_user_actor() |> Repo.all() end - @spec get_reported_status_ids() :: %{ - required(:items) => [%Activity{}], - required(:total) => integer - } + @spec get_reported_status_ids() :: [ + %{ + required(:activity) => String.t(), + required(:date) => String.t() + } + ] def get_reported_status_ids do from(a in Activity, where: fragment("(?)->>'type' = 'Flag'", a.data), select: %{ date: fragment("max(?->>'published') date", a.data), - activity_id: - fragment("jsonb_array_elements_text((? #- '{object,0}')->'object') activity_id", a.data) + activity: + fragment("jsonb_array_elements_text((? #- '{object,0}')->'object') activity", a.data) }, - group_by: fragment("activity_id"), + group_by: fragment("activity"), order_by: fragment("date DESC") ) |> Repo.all() diff --git a/lib/pleroma/web/admin_api/views/report_view.ex b/lib/pleroma/web/admin_api/views/report_view.ex index ac25925da..ca88595c7 100644 --- a/lib/pleroma/web/admin_api/views/report_view.ex +++ b/lib/pleroma/web/admin_api/views/report_view.ex @@ -47,8 +47,8 @@ def render("index_grouped.json", %{groups: groups}) do Enum.map(groups, fn group -> %{ date: group[:date], - account: merge_account_views(group[:account]), - status: StatusView.render("show.json", %{activity: group[:status]}), + account: group[:account], + status: group[:status], actors: Enum.map(group[:actors], &merge_account_views/1), reports: group[:reports] diff --git a/test/web/admin_api/admin_api_controller_test.exs b/test/web/admin_api/admin_api_controller_test.exs index 35367bed3..4e28c7774 100644 --- a/test/web/admin_api/admin_api_controller_test.exs +++ b/test/web/admin_api/admin_api_controller_test.exs @@ -1551,7 +1551,7 @@ test "returns 403 when requested by anonymous" do describe "GET /api/pleroma/admin/grouped_reports" do setup %{conn: conn} do - admin = insert(:user, info: %{is_admin: true}) + admin = insert(:user, is_admin: true) [reporter, target_user] = insert_pair(:user) date1 = (DateTime.to_unix(DateTime.utc_now()) + 1000) |> DateTime.from_unix!() @@ -1567,53 +1567,124 @@ test "returns 403 when requested by anonymous" do third_status = insert(:note_activity, user: target_user, data_attrs: %{"published" => date3}) - %{ - conn: assign(conn, :user, admin), - reporter: reporter, - target_user: target_user, - first_status: first_status, - second_status: second_status, - third_status: third_status - } - end - - test "returns reports grouped by status", %{ - conn: conn, - reporter: reporter, - target_user: target_user, - first_status: first_status, - second_status: second_status, - third_status: third_status - } do - {:ok, %{id: _}} = + {:ok, first_report} = CommonAPI.report(reporter, %{ "account_id" => target_user.id, "status_ids" => [first_status.id, second_status.id, third_status.id] }) - {:ok, %{id: _}} = + {:ok, second_report} = CommonAPI.report(reporter, %{ "account_id" => target_user.id, "status_ids" => [first_status.id, second_status.id] }) - {:ok, %{id: _}} = + {:ok, third_report} = CommonAPI.report(reporter, %{ "account_id" => target_user.id, "status_ids" => [first_status.id] }) + %{ + conn: assign(conn, :user, admin), + first_status: Activity.get_by_ap_id_with_object(first_status.data["id"]), + second_status: Activity.get_by_ap_id_with_object(second_status.data["id"]), + third_status: Activity.get_by_ap_id_with_object(third_status.data["id"]), + first_status_reports: [first_report, second_report, third_report], + second_status_reports: [first_report, second_report], + third_status_reports: [first_report], + target_user: target_user, + reporter: reporter + } + end + + test "returns reports grouped by status", %{ + conn: conn, + first_status: first_status, + second_status: second_status, + third_status: third_status, + first_status_reports: first_status_reports, + second_status_reports: second_status_reports, + third_status_reports: third_status_reports, + target_user: target_user, + reporter: reporter + } do response = conn |> get("/api/pleroma/admin/grouped_reports") |> json_response(:ok) assert length(response["reports"]) == 3 - [third_group, second_group, first_group] = response["reports"] - assert length(third_group["reports"]) == 3 + first_group = + Enum.find(response["reports"], &(&1["status"]["id"] == first_status.data["id"])) + + second_group = + Enum.find(response["reports"], &(&1["status"]["id"] == second_status.data["id"])) + + third_group = + Enum.find(response["reports"], &(&1["status"]["id"] == third_status.data["id"])) + + assert length(first_group["reports"]) == 3 assert length(second_group["reports"]) == 2 - assert length(first_group["reports"]) == 1 + assert length(third_group["reports"]) == 1 + + assert first_group["date"] == + Enum.max_by(first_status_reports, fn act -> + NaiveDateTime.from_iso8601!(act.data["published"]) + end).data["published"] + + assert first_group["status"] == %{ + "id" => first_status.data["id"], + "content" => first_status.object.data["content"], + "published" => first_status.object.data["published"] + } + + assert first_group["account"]["id"] == target_user.id + + assert length(first_group["actors"]) == 1 + assert hd(first_group["actors"])["id"] == reporter.id + + assert Enum.map(first_group["reports"], & &1["id"]) -- + Enum.map(first_status_reports, & &1.id) == [] + + assert second_group["date"] == + Enum.max_by(second_status_reports, fn act -> + NaiveDateTime.from_iso8601!(act.data["published"]) + end).data["published"] + + assert second_group["status"] == %{ + "id" => second_status.data["id"], + "content" => second_status.object.data["content"], + "published" => second_status.object.data["published"] + } + + assert second_group["account"]["id"] == target_user.id + + assert length(second_group["actors"]) == 1 + assert hd(second_group["actors"])["id"] == reporter.id + + assert Enum.map(second_group["reports"], & &1["id"]) -- + Enum.map(second_status_reports, & &1.id) == [] + + assert third_group["date"] == + Enum.max_by(third_status_reports, fn act -> + NaiveDateTime.from_iso8601!(act.data["published"]) + end).data["published"] + + assert third_group["status"] == %{ + "id" => third_status.data["id"], + "content" => third_status.object.data["content"], + "published" => third_status.object.data["published"] + } + + assert third_group["account"]["id"] == target_user.id + + assert length(third_group["actors"]) == 1 + assert hd(third_group["actors"])["id"] == reporter.id + + assert Enum.map(third_group["reports"], & &1["id"]) -- + Enum.map(third_status_reports, & &1.id) == [] end end From 7258db023e88d5aee5eac06525c42dcb073abd46 Mon Sep 17 00:00:00 2001 From: Maxim Filippov Date: Thu, 7 Nov 2019 22:45:36 +1000 Subject: [PATCH 6/9] Support old flag format --- lib/pleroma/web/activity_pub/utils.ex | 94 +++++++++++-------- .../web/admin_api/admin_api_controller.ex | 4 +- test/web/activity_pub/utils_test.exs | 43 +++++++++ 3 files changed, 100 insertions(+), 41 deletions(-) diff --git a/lib/pleroma/web/activity_pub/utils.ex b/lib/pleroma/web/activity_pub/utils.ex index 5a51b7884..5e7f76bb6 100644 --- a/lib/pleroma/web/activity_pub/utils.ex +++ b/lib/pleroma/web/activity_pub/utils.ex @@ -616,26 +616,31 @@ def make_flag_data(%{actor: actor, context: context, content: content} = params, def make_flag_data(_, _), do: %{} defp build_flag_object(%{account: account, statuses: statuses} = _) do - [account.ap_id] ++ - Enum.map(statuses || [], fn act -> - id = - case act do - %Activity{} = act -> act.data["id"] - act when is_map(act) -> act["id"] - act when is_binary(act) -> act - end + [account.ap_id] ++ build_flag_object(%{statuses: statuses}) + end - activity = Activity.get_by_ap_id_with_object(id) - actor = User.get_by_ap_id(activity.object.data["actor"]) + defp build_flag_object(%{statuses: statuses}) do + Enum.map(statuses || [], &build_flag_object/1) + end - %{ - "type" => "Note", - "id" => activity.data["id"], - "content" => activity.object.data["content"], - "published" => activity.object.data["published"], - "actor" => AccountView.render("show.json", %{user: actor}) - } - end) + defp build_flag_object(act) when is_map(act) or is_binary(act) do + id = + case act do + %Activity{} = act -> act.data["id"] + act when is_map(act) -> act["id"] + act when is_binary(act) -> act + end + + activity = Activity.get_by_ap_id_with_object(id) + actor = User.get_by_ap_id(activity.object.data["actor"]) + + %{ + "type" => "Note", + "id" => activity.data["id"], + "content" => activity.object.data["content"], + "published" => activity.object.data["published"], + "actor" => AccountView.render("show.json", %{user: actor}) + } end defp build_flag_object(_), do: [] @@ -692,7 +697,7 @@ def get_reports(params, page, page_size) do ActivityPub.fetch_activities([], params, :offset) end - @spec get_reports_grouped_by_status() :: %{ + @spec get_reports_grouped_by_status(%{required(:activity) => String.t()}) :: %{ required(:groups) => [ %{ required(:date) => String.t(), @@ -704,30 +709,39 @@ def get_reports(params, page, page_size) do ], required(:total) => integer } - def get_reports_grouped_by_status do - groups = - get_reported_status_ids() + def get_reports_grouped_by_status(groups) do + parsed_groups = + groups |> Enum.map(fn entry -> - activity = Jason.decode!(entry.activity) - reports = get_reports_by_status_id(activity["id"]) - max_date = Enum.max_by(reports, &NaiveDateTime.from_iso8601!(&1.data["published"])) - actors = Enum.map(reports, & &1.user_actor) + activity = + case Jason.decode(entry.activity) do + {:ok, activity} -> activity + _ -> build_flag_object(entry.activity) + end - %{ - date: max_date.data["published"], - account: activity["actor"], - status: %{ - id: activity["id"], - content: activity["content"], - published: activity["published"] - }, - actors: Enum.uniq(actors), - reports: reports - } + parse_report_group(activity) end) %{ - groups: groups + groups: parsed_groups + } + end + + def parse_report_group(activity) do + reports = get_reports_by_status_id(activity["id"]) + max_date = Enum.max_by(reports, &NaiveDateTime.from_iso8601!(&1.data["published"])) + actors = Enum.map(reports, & &1.user_actor) + + %{ + date: max_date.data["published"], + account: activity["actor"], + status: %{ + id: activity["id"], + content: activity["content"], + published: activity["published"] + }, + actors: Enum.uniq(actors), + reports: reports } end @@ -740,13 +754,13 @@ def get_reports_by_status_id(ap_id) do |> Repo.all() end - @spec get_reported_status_ids() :: [ + @spec get_reported_activities() :: [ %{ required(:activity) => String.t(), required(:date) => String.t() } ] - def get_reported_status_ids do + def get_reported_activities do from(a in Activity, where: fragment("(?)->>'type' = 'Flag'", a.data), select: %{ diff --git a/lib/pleroma/web/admin_api/admin_api_controller.ex b/lib/pleroma/web/admin_api/admin_api_controller.ex index 1f48ce8c1..7d5ff7629 100644 --- a/lib/pleroma/web/admin_api/admin_api_controller.ex +++ b/lib/pleroma/web/admin_api/admin_api_controller.ex @@ -625,9 +625,11 @@ def list_reports(conn, params) do end def list_grouped_reports(conn, _params) do + reports = Utils.get_reported_activities() + conn |> put_view(ReportView) - |> render("index_grouped.json", Utils.get_reports_grouped_by_status()) + |> render("index_grouped.json", Utils.get_reports_grouped_by_status(reports)) end def report_show(conn, %{"id" => id}) do diff --git a/test/web/activity_pub/utils_test.exs b/test/web/activity_pub/utils_test.exs index 586eb1d2f..1feb076ba 100644 --- a/test/web/activity_pub/utils_test.exs +++ b/test/web/activity_pub/utils_test.exs @@ -636,4 +636,47 @@ test "removes actor from announcements" do assert updated_object.data["announcement_count"] == 1 end end + + describe "get_reports_grouped_by_status/1" do + setup do + [reporter, target_user] = insert_pair(:user) + first_status = insert(:note_activity, user: target_user) + second_status = insert(:note_activity, user: target_user) + + CommonAPI.report(reporter, %{ + "account_id" => target_user.id, + "comment" => "I feel offended", + "status_ids" => [first_status.id] + }) + + CommonAPI.report(reporter, %{ + "account_id" => target_user.id, + "comment" => "I feel offended2", + "status_ids" => [second_status.id] + }) + + data = [%{activity: first_status.data["id"]}, %{activity: second_status.data["id"]}] + + {:ok, + %{ + first_status: first_status, + second_status: second_status, + data: data + }} + end + + test "works for deprecated reports format", %{ + first_status: first_status, + second_status: second_status, + data: data + } do + groups = Utils.get_reports_grouped_by_status(data).groups + + first_group = Enum.find(groups, &(&1.status.id == first_status.data["id"])) + second_group = Enum.find(groups, &(&1.status.id == second_status.data["id"])) + + assert first_group.status.id == first_status.data["id"] + assert second_group.status.id == second_status.data["id"] + end + end end From b2e8371e6a366a1b75c520c617a23edfff3f1274 Mon Sep 17 00:00:00 2001 From: Maxim Filippov Date: Mon, 11 Nov 2019 09:55:00 +0000 Subject: [PATCH 7/9] Apply suggestion to CHANGELOG.md --- CHANGELOG.md | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8e03ee6b5..b33d61819 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -39,7 +39,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Authentication: Added rate limit for password-authorized actions / login existence checks - Mix task to re-count statuses for all users (`mix pleroma.count_statuses`) - Support for `X-Forwarded-For` and similar HTTP headers which used by reverse proxies to pass a real user IP address to the backend. Must not be enabled unless your instance is behind at least one reverse proxy (such as Nginx, Apache HTTPD or Varnish Cache). -- Admin API: Add ability to fetch reports, grouped by status `GET /api/pleroma/admin/grouped_reports`
API Changes From 31343e4321a3c4053b66a1d6dc3da0e42dbdd972 Mon Sep 17 00:00:00 2001 From: Maxim Filippov Date: Mon, 11 Nov 2019 19:06:09 +0900 Subject: [PATCH 8/9] Code style fixes --- CHANGELOG.md | 1 + docs/API/admin_api.md | 2 -- lib/pleroma/activity.ex | 7 +------ mix.lock | 2 +- 4 files changed, 3 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b33d61819..5442bfc5e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -43,6 +43,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). API Changes - Job queue stats to the healthcheck page +- Admin API: Add ability to fetch reports, grouped by status `GET /api/pleroma/admin/grouped_reports` - Admin API: Add ability to require password reset - Mastodon API: Account entities now include `follow_requests_count` (planned Mastodon 3.x addition) - Pleroma API: `GET /api/v1/pleroma/accounts/:id/scrobbles` to get a list of recently scrobbled items diff --git a/docs/API/admin_api.md b/docs/API/admin_api.md index ce70b5122..9d914c9a6 100644 --- a/docs/API/admin_api.md +++ b/docs/API/admin_api.md @@ -58,7 +58,6 @@ Authentication is required and the user must be an admin. ### Remove a user -- Method `DELETE` - Params: - `nicknames` - Response: Array of user nicknames @@ -735,7 +734,6 @@ Copy settings on key `:pleroma` to DB. Copy all settings from DB to `config/prod.exported_from_db.secret.exs` with deletion from DB. -- Method `GET` - Params: none - Response: diff --git a/lib/pleroma/activity.ex b/lib/pleroma/activity.ex index 7b77f72c2..7e283df32 100644 --- a/lib/pleroma/activity.ex +++ b/lib/pleroma/activity.ex @@ -92,12 +92,7 @@ def with_preloaded_object(query, join_type \\ :inner) do def with_joined_user_actor(query, join_type \\ :inner) do join(query, join_type, [activity], u in User, - on: - fragment( - "? = ?->>'actor'", - u.ap_id, - activity.data - ), + on: u.ap_id == activity.actor, as: :user_actor ) end diff --git a/mix.lock b/mix.lock index c707667b2..4529506a8 100644 --- a/mix.lock +++ b/mix.lock @@ -36,7 +36,7 @@ "ex_rated": {:hex, :ex_rated, "1.3.3", "30ecbdabe91f7eaa9d37fa4e81c85ba420f371babeb9d1910adbcd79ec798d27", [:mix], [{:ex2ms, "~> 1.5", [hex: :ex2ms, repo: "hexpm", optional: false]}], "hexpm"}, "ex_syslogger": {:git, "https://github.com/slashmili/ex_syslogger.git", "f3963399047af17e038897c69e20d552e6899e1d", [tag: "1.4.0"]}, "excoveralls": {:hex, :excoveralls, "0.11.2", "0c6f2c8db7683b0caa9d490fb8125709c54580b4255ffa7ad35f3264b075a643", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm"}, - "fast_html": {:hex, :fast_html, "0.99.0", "ea740358b15c7da6085b421b775f22d4f2c6928a28a15ebb5ad4e8a2ce00350b", [:make, :mix], [], "hexpm"}, + "fast_html": {:hex, :fast_html, "0.99.3", "e7ce6245fed0635f4719a31cc409091ed17b2091165a4a1cffbf2ceac77abbf4", [:make, :mix], [], "hexpm"}, "fast_sanitize": {:hex, :fast_sanitize, "0.1.1", "a403c3c09369e23423d3e6beb14068ad07be82741d10b293c71abac445dcc636", [:mix], [{:fast_html, "~> 0.99", [hex: :fast_html, repo: "hexpm", optional: false]}, {:plug, "~> 1.8", [hex: :plug, repo: "hexpm", optional: false]}], "hexpm"}, "flake_id": {:hex, :flake_id, "0.1.0", "7716b086d2e405d09b647121a166498a0d93d1a623bead243e1f74216079ccb3", [:mix], [{:base62, "~> 1.2", [hex: :base62, repo: "hexpm", optional: false]}, {:ecto, ">= 2.0.0", [hex: :ecto, repo: "hexpm", optional: true]}], "hexpm"}, "floki": {:hex, :floki, "0.23.0", "956ab6dba828c96e732454809fb0bd8d43ce0979b75f34de6322e73d4c917829", [:mix], [{:html_entities, "~> 0.4.0", [hex: :html_entities, repo: "hexpm", optional: false]}], "hexpm"}, From 1649d6f6894bbb2c36095d34eddd17d2e5f8d9df Mon Sep 17 00:00:00 2001 From: Maxim Filippov Date: Mon, 11 Nov 2019 19:16:04 +0900 Subject: [PATCH 9/9] Add "/api/pleroma/admin/reports/:id" -> "/api/pleroma/admin/reports" changelog entry --- CHANGELOG.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5442bfc5e..411b0b46d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -24,6 +24,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - **Breaking** Admin API: `PATCH /api/pleroma/admin/users/:nickname/force_password_reset` is now `PATCH /api/pleroma/admin/users/force_password_reset` (accepts `nicknames` array in the request body) - **Breaking:** Admin API: Return link alongside with token on password reset +- **Breaking:** Admin API: `PUT /api/pleroma/admin/reports/:id` is now `PATCH /api/pleroma/admin/reports`, see admin_api.md for details - **Breaking:** `/api/pleroma/admin/users/invite_token` now uses `POST`, changed accepted params and returns full invite in json instead of only token string. - Admin API: Return `total` when querying for reports - Mastodon API: Return `pleroma.direct_conversation_id` when creating a direct message (`POST /api/v1/statuses`) @@ -54,7 +55,9 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/). - Metadata Link: Atom syndication Feed - Mastodon API: Add `exclude_visibilities` parameter to the timeline and notification endpoints - Admin API: `/users/:nickname/toggle_activation` endpoint is now deprecated in favor of: `/users/activate`, `/users/deactivate`, both accept `nicknames` array -- Admin API: `POST/DELETE /api/pleroma/admin/users/:nickname/permission_group/:permission_group` are deprecated in favor of: `POST/DELETE /api/pleroma/admin/users/permission_group/:permission_group` (both accept `nicknames` array), `DELETE /api/pleroma/admin/users` (`nickname` query param or `nickname` sent in JSON body) is deprecated in favor of: `DELETE /api/pleroma/admin/users` (`nicknames` query array param or `nicknames` sent in JSON body). +- Admin API: Multiple endpoints now require `nicknames` array, instead of singe `nickname`: + - `POST/DELETE /api/pleroma/admin/users/:nickname/permission_group/:permission_group` are deprecated in favor of: `POST/DELETE /api/pleroma/admin/users/permission_group/:permission_group` + - `DELETE /api/pleroma/admin/users` (`nickname` query param or `nickname` sent in JSON body) is deprecated in favor of: `DELETE /api/pleroma/admin/users` (`nicknames` query array param or `nicknames` sent in JSON body) - Admin API: Add `GET /api/pleroma/admin/relay` endpoint - lists all followed relays - Pleroma API: `POST /api/v1/pleroma/conversations/read` to mark all conversations as read - Mastodon API: Add `/api/v1/markers` for managing timeline read markers