Theming and such

Update to 4.1.3
Bump version to v4.1.3
2023-07-06 13:23:31 -04:00 · 2023-07-06 09:38:36 -04:00 · 2023-07-06 15:07:20 +02:00 · 2023-07-06 15:06:50 +02:00 · 2023-07-06 15:06:24 +02:00 · 2023-07-06 15:05:05 +02:00
37 changed files with 5118 additions and 104 deletions
--- a/.prettierignore
+++ b/.prettierignore
@ -70,3 +70,23 @@ docker-compose.override.yml
 # Ignore locale files
 /app/javascript/mastodon/locales
 /config/locales
+<<<<<<< Updated upstream
+=======
+
+# Ignore vendored CSS reset
+app/javascript/styles/mastodon/reset.scss
+
+# Ignore Javascript pending https://github.com/mastodon/mastodon/pull/23631
+*.js
+*.jsx
+*.ts
+*.tsx
+
+# Ignore HTML till cleaned and included in CI
+*.html
+
+# Ignore the generated AUTHORS.md
+AUTHORS.md
+
+/Mastodon-Modern
+>>>>>>> Stashed changes
--- a/.stylelintignore
+++ b/.stylelintignore
@ -0,0 +1 @@
+app/javascript/styles/modern/style.scss
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@ -3,6 +3,54 @@ Changelog

 All notable changes to this project will be documented in this file.

+## [4.1.3] - 2023-07-06
+
+### Added
+
+- Add fallback redirection when getting a webfinger query `LOCAL_DOMAIN@LOCAL_DOMAIN` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/23600))
+
+### Changed
+
+- Change OpenGraph-based embeds to allow fullscreen ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25058))
+- Change AccessTokensVacuum to also delete expired tokens ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24868))
+- Change profile updates to be sent to recently-mentioned servers ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24852))
+- Change automatic post deletion thresholds and load detection ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24614))
+- Change `/api/v1/statuses/:id/history` to always return at least one item ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25510))
+- Change auto-linking to allow carets in URL query params ([renchap](https://github.com/mastodon/mastodon/pull/25216))
+
+### Removed
+
+- Remove invalid `X-Frame-Options: ALLOWALL` ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25070))
+
+### Fixed
+
+- Fix wrong view being displayed when a webhook fails validation ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25464))
+- Fix soft-deleted post cleanup scheduler overwhelming the streaming server ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/25519))
+- Fix incorrect pagination headers in `/api/v2/admin/accounts` ([danielmbrasil](https://github.com/mastodon/mastodon/pull/25477))
+- Fix multiple inefficiencies in automatic post cleanup worker ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24607), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24785), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/24840))
+- Fix performance of streaming by parsing message JSON once ([ThisIsMissEm](https://github.com/mastodon/mastodon/pull/25278), [ThisIsMissEm](https://github.com/mastodon/mastodon/pull/25361))
+- Fix CSP headers when `S3_ALIAS_HOST` includes a path component ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25273))
+- Fix `tootctl accounts approve --number N` not aproving N earliest registrations ([danielmbrasil](https://github.com/mastodon/mastodon/pull/24605))
+- Fix reports not being closed when performing batch suspensions ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24988))
+- Fix being able to vote on your own polls ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25015))
+- Fix race condition when reblogging a status ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25016))
+- Fix “Authorized applications” inefficiently and incorrectly getting last use date ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25060))
+- Fix “Authorized applications” crashing when listing apps with certain admin API scopes ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25713))
+- Fix multiple N+1s in ConversationsController ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25134), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25399), [ClearlyClaire](https://github.com/mastodon/mastodon/pull/25499))
+- Fix user archive takeouts when using OpenStack Swift ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/24431))
+- Fix searching for remote content by URL not working under certain conditions ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25637))
+- Fix inefficiencies in indexing content for search ([VyrCossont](https://github.com/mastodon/mastodon/pull/24285), [VyrCossont](https://github.com/mastodon/mastodon/pull/24342))
+
+### Security
+
+- Add finer permission requirements for managing webhooks ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25463))
+- Update dependencies
+- Add hardening headers for user-uploaded files ([ClearlyClaire](https://github.com/mastodon/mastodon/pull/25756))
+- Fix verified links possibly hiding important parts of the URL (CVE-2023-36462)
+- Fix timeout handling of outbound HTTP requests (CVE-2023-36461)
+- Fix arbitrary file creation through media processing (CVE-2023-36460)
+- Fix possible XSS in preview cards (CVE-2023-36459)
+
 ## [4.1.2] - 2023-04-04

 ### Fixed
--- a/Gemfile.lock
+++ b/Gemfile.lock
@ -10,40 +10,40 @@ GIT
 GEM
  remote: https://rubygems.org/
  specs:
-    actioncable (6.1.7.2)
-      actionpack (= 6.1.7.2)
-      activesupport (= 6.1.7.2)
+    actioncable (6.1.7.4)
+      actionpack (= 6.1.7.4)
+      activesupport (= 6.1.7.4)
      nio4r (~> 2.0)
      websocket-driver (>= 0.6.1)
-    actionmailbox (6.1.7.2)
-      actionpack (= 6.1.7.2)
-      activejob (= 6.1.7.2)
-      activerecord (= 6.1.7.2)
-      activestorage (= 6.1.7.2)
-      activesupport (= 6.1.7.2)
+    actionmailbox (6.1.7.4)
+      actionpack (= 6.1.7.4)
+      activejob (= 6.1.7.4)
+      activerecord (= 6.1.7.4)
+      activestorage (= 6.1.7.4)
+      activesupport (= 6.1.7.4)
      mail (>= 2.7.1)
-    actionmailer (6.1.7.2)
-      actionpack (= 6.1.7.2)
-      actionview (= 6.1.7.2)
-      activejob (= 6.1.7.2)
-      activesupport (= 6.1.7.2)
+    actionmailer (6.1.7.4)
+      actionpack (= 6.1.7.4)
+      actionview (= 6.1.7.4)
+      activejob (= 6.1.7.4)
+      activesupport (= 6.1.7.4)
      mail (~> 2.5, >= 2.5.4)
      rails-dom-testing (~> 2.0)
-    actionpack (6.1.7.2)
-      actionview (= 6.1.7.2)
-      activesupport (= 6.1.7.2)
+    actionpack (6.1.7.4)
+      actionview (= 6.1.7.4)
+      activesupport (= 6.1.7.4)
      rack (~> 2.0, >= 2.0.9)
      rack-test (>= 0.6.3)
      rails-dom-testing (~> 2.0)
      rails-html-sanitizer (~> 1.0, >= 1.2.0)
-    actiontext (6.1.7.2)
-      actionpack (= 6.1.7.2)
-      activerecord (= 6.1.7.2)
-      activestorage (= 6.1.7.2)
-      activesupport (= 6.1.7.2)
+    actiontext (6.1.7.4)
+      actionpack (= 6.1.7.4)
+      activerecord (= 6.1.7.4)
+      activestorage (= 6.1.7.4)
+      activesupport (= 6.1.7.4)
      nokogiri (>= 1.8.5)
-    actionview (6.1.7.2)
-      activesupport (= 6.1.7.2)
+    actionview (6.1.7.4)
+      activesupport (= 6.1.7.4)
      builder (~> 3.1)
      erubi (~> 1.4)
      rails-dom-testing (~> 2.0)
@ -54,22 +54,22 @@ GEM
      case_transform (>= 0.2)
      jsonapi-renderer (>= 0.1.1.beta1, < 0.3)
    active_record_query_trace (1.8)
-    activejob (6.1.7.2)
-      activesupport (= 6.1.7.2)
+    activejob (6.1.7.4)
+      activesupport (= 6.1.7.4)
      globalid (>= 0.3.6)
-    activemodel (6.1.7.2)
-      activesupport (= 6.1.7.2)
-    activerecord (6.1.7.2)
-      activemodel (= 6.1.7.2)
-      activesupport (= 6.1.7.2)
-    activestorage (6.1.7.2)
-      actionpack (= 6.1.7.2)
-      activejob (= 6.1.7.2)
-      activerecord (= 6.1.7.2)
-      activesupport (= 6.1.7.2)
+    activemodel (6.1.7.4)
+      activesupport (= 6.1.7.4)
+    activerecord (6.1.7.4)
+      activemodel (= 6.1.7.4)
+      activesupport (= 6.1.7.4)
+    activestorage (6.1.7.4)
+      actionpack (= 6.1.7.4)
+      activejob (= 6.1.7.4)
+      activerecord (= 6.1.7.4)
+      activesupport (= 6.1.7.4)
      marcel (~> 1.0)
      mini_mime (>= 1.1.0)
-    activesupport (6.1.7.2)
+    activesupport (6.1.7.4)
      concurrent-ruby (~> 1.0, >= 1.0.2)
      i18n (>= 1.6, < 2)
      minitest (>= 5.1)
@ -173,7 +173,7 @@ GEM
    cocoon (1.2.15)
    coderay (1.1.3)
    color_diff (0.1)
-    concurrent-ruby (1.2.0)
+    concurrent-ruby (1.2.2)
    connection_pool (2.3.0)
    cose (1.2.1)
      cbor (~> 0.5.9)
@ -206,7 +206,7 @@ GEM
    docile (1.4.0)
    domain_name (0.5.20190701)
      unf (>= 0.0.5, < 1.0.0)
-    doorkeeper (5.6.4)
+    doorkeeper (5.6.6)
      railties (>= 5)
    dotenv (2.8.1)
    dotenv-rails (2.8.1)
@ -388,7 +388,7 @@ GEM
    loofah (2.19.1)
      crass (~> 1.0.2)
      nokogiri (>= 1.5.9)
-    mail (2.8.0.1)
+    mail (2.8.1)
      mini_mime (>= 0.1.1)
      net-imap
      net-pop
@ -405,12 +405,12 @@ GEM
      mime-types-data (~> 3.2015)
    mime-types-data (3.2022.0105)
    mini_mime (1.1.2)
-    mini_portile2 (2.8.1)
+    mini_portile2 (2.8.2)
    minitest (5.17.0)
    msgpack (1.6.0)
    multi_json (1.15.0)
    multipart-post (2.1.1)
-    net-imap (0.3.4)
+    net-imap (0.3.6)
      date
      net-protocol
    net-ldap (0.17.1)
@ -423,8 +423,8 @@ GEM
    net-smtp (0.3.3)
      net-protocol
    net-ssh (7.0.1)
-    nio4r (2.5.8)
-    nokogiri (1.14.1)
+    nio4r (2.5.9)
+    nokogiri (1.14.5)
      mini_portile2 (~> 2.8.0)
      racc (~> 1.4)
    nsa (0.2.8)
@ -497,7 +497,7 @@ GEM
      activesupport (>= 3.0.0)
    raabro (1.4.0)
    racc (1.6.2)
-    rack (2.2.6.2)
+    rack (2.2.7)
    rack-attack (6.6.1)
      rack (>= 1.0, < 3)
    rack-cors (1.1.1)
@ -512,20 +512,20 @@ GEM
      rack
    rack-test (2.0.2)
      rack (>= 1.3)
-    rails (6.1.7.2)
-      actioncable (= 6.1.7.2)
-      actionmailbox (= 6.1.7.2)
-      actionmailer (= 6.1.7.2)
-      actionpack (= 6.1.7.2)
-      actiontext (= 6.1.7.2)
-      actionview (= 6.1.7.2)
-      activejob (= 6.1.7.2)
-      activemodel (= 6.1.7.2)
-      activerecord (= 6.1.7.2)
-      activestorage (= 6.1.7.2)
-      activesupport (= 6.1.7.2)
+    rails (6.1.7.4)
+      actioncable (= 6.1.7.4)
+      actionmailbox (= 6.1.7.4)
+      actionmailer (= 6.1.7.4)
+      actionpack (= 6.1.7.4)
+      actiontext (= 6.1.7.4)
+      actionview (= 6.1.7.4)
+      activejob (= 6.1.7.4)
+      activemodel (= 6.1.7.4)
+      activerecord (= 6.1.7.4)
+      activestorage (= 6.1.7.4)
+      activesupport (= 6.1.7.4)
      bundler (>= 1.15.0)
-      railties (= 6.1.7.2)
+      railties (= 6.1.7.4)
      sprockets-rails (>= 2.0.0)
    rails-controller-testing (1.0.5)
      actionpack (>= 5.0.1.rc1)
@ -541,9 +541,9 @@ GEM
      railties (>= 6.0.0, < 7)
    rails-settings-cached (0.6.6)
      rails (>= 4.2.0)
-    railties (6.1.7.2)
-      actionpack (= 6.1.7.2)
-      activesupport (= 6.1.7.2)
+    railties (6.1.7.4)
+      actionpack (= 6.1.7.4)
+      activesupport (= 6.1.7.4)
      method_source
      rake (>= 12.2)
      thor (~> 1.0)
@ -688,9 +688,9 @@ GEM
      unicode-display_width (>= 1.1.1, < 3)
    terrapin (0.6.0)
      climate_control (>= 0.0.3, < 1.0)
-    thor (1.2.1)
+    thor (1.2.2)
    tilt (2.0.11)
-    timeout (0.3.1)
+    timeout (0.3.2)
    tpm-key_attestation (0.11.0)
      bindata (~> 2.4)
      openssl (> 2.0, < 3.1)
@ -753,7 +753,7 @@ GEM
    xorcist (1.1.3)
    xpath (3.2.0)
      nokogiri (~> 1.8)
-    zeitwerk (2.6.6)
+    zeitwerk (2.6.8)

 PLATFORMS
  ruby
--- a/1
+++ b/1
@ -0,0 +1 @@
+Subproject commit 301d0033f593a35a289a22f1891a0dfe12245d7f
--- a/app/helpers/formatting_helper.rb
+++ b/app/helpers/formatting_helper.rb
@ -58,6 +58,10 @@ module FormattingHelper
  end

  def account_field_value_format(field, with_rel_me: true)
-    html_aware_format(field.value, field.account.local?, with_rel_me: with_rel_me, with_domains: true, multiline: false)
+    if field.verified? && !field.account.local?
+      TextFormatter.shortened_link(field.value_for_verification)
+    else
+      html_aware_format(field.value, field.account.local?, with_rel_me: with_rel_me, with_domains: true, multiline: false)
+    end
  end
 end
--- a/app/javascript/mastodon/features/compose/components/compose_form.js
+++ b/app/javascript/mastodon/features/compose/components/compose_form.js
@ -90,7 +90,7 @@ class ComposeForm extends ImmutablePureComponent {
    const fulltext = this.getFulltextForCharacterCounting();
    const isOnlyWhitespace = fulltext.length !== 0 && fulltext.trim().length === 0;

-    return !(isSubmitting || isUploading || isChangingUpload || length(fulltext) > 500 || (isOnlyWhitespace && !anyMedia));
+    return !(isSubmitting || isUploading || isChangingUpload || length(fulltext) > 1500 || (isOnlyWhitespace && !anyMedia));
  };

  handleSubmit = (e) => {
@ -280,7 +280,7 @@ class ComposeForm extends ImmutablePureComponent {
          </div>

          <div className='character-counter__wrapper'>
-            <CharacterCounter max={500} text={this.getFulltextForCharacterCounting()} />
+            <CharacterCounter max={1500} text={this.getFulltextForCharacterCounting()} />
          </div>
        </div>

--- a/app/javascript/styles/modern-dark-wobbl.scss
+++ b/app/javascript/styles/modern-dark-wobbl.scss
@ -0,0 +1,8 @@
+@import 'wobbl/variables';
+@import 'application';
+@import 'modern/modern';
+
+.layout-multiple-columns .column {
+	flex-grow: 1;
+	max-width: 500px;
+}
--- a/app/javascript/styles/modern-dark.scss
+++ b/app/javascript/styles/modern-dark.scss
@ -0,0 +1,8 @@
+@import 'mastodon/variables';
+@import 'application';
+@import 'modern/modern';
+
+.layout-multiple-columns .column {
+	flex-grow: 1;
+	max-width: 500px;
+}
--- a/app/javascript/styles/modern-light-wobbl.scss
+++ b/app/javascript/styles/modern-light-wobbl.scss
@ -0,0 +1,9 @@
+@import 'wobbl-light/variables';
+@import 'application';
+@import 'modern/modern';
+@import 'mastodon-light/diff';
+
+.layout-multiple-columns .column {
+	flex-grow: 1;
+	max-width: 500px;
+}
--- a/app/javascript/styles/modern-light.scss
+++ b/app/javascript/styles/modern-light.scss
@ -0,0 +1,9 @@
+@import 'mastodon-light/variables';
+@import 'application';
+@import 'modern/modern';
+@import 'mastodon-light/diff';
+
+.layout-multiple-columns .column {
+	flex-grow: 1;
+	max-width: 500px;
+}
--- a/app/javascript/styles/modern/modern.scss
+++ b/app/javascript/styles/modern/modern.scss
--- a/app/javascript/styles/wobbl-light.scss
+++ b/app/javascript/styles/wobbl-light.scss
@ -0,0 +1,4 @@
+@import 'wobbl-light/variables';
+@import 'application';
+@import 'mastodon-light/diff';
+
--- a/app/javascript/styles/wobbl-light/variables.scss
+++ b/app/javascript/styles/wobbl-light/variables.scss
@ -0,0 +1,44 @@
+// Dependent colors
+$black: #000000;
+$white: #ffffff;
+
+$classic-base-color: #191919;// Dark Gray
+$classic-primary-color: #E7E7E7;// Platinum
+$classic-secondary-color: #FF8680;    // Paler red
+$classic-highlight-color: #850700;    // Red highlights
+
+// Differences
+$success-green: lighten(#3c754d, 8%);
+
+$base-overlay-background: $white !default;
+$valid-value-color: $success-green !default;
+
+$ui-base-color: $classic-primary-color !default;
+$ui-base-lighter-color: #b0c0cf;
+$ui-primary-color: #9bcbed;
+$ui-secondary-color: $classic-base-color !default;
+$ui-highlight-color: $classic-highlight-color !default;
+
+$primary-text-color: $black !default;
+$darker-text-color: $classic-base-color !default;
+$highlight-text-color: darken($ui-highlight-color, 8%) !default;
+$dark-text-color: #444b5d;
+$action-button-color: #606984;
+
+$inverted-text-color: $black !default;
+$lighter-text-color: $classic-base-color !default;
+$light-text-color: #444b5d;
+
+// Newly added colors
+$account-background-color: $white !default;
+
+// Invert darkened and lightened colors
+@function darken($color, $amount) {
+  @return hsl(hue($color), saturation($color), lightness($color) + $amount);
+}
+
+@function lighten($color, $amount) {
+  @return hsl(hue($color), saturation($color), lightness($color) - $amount);
+}
+
+$emojis-requiring-inversion: 'chains';
--- a/app/javascript/styles/wobbl.scss
+++ b/app/javascript/styles/wobbl.scss
@ -0,0 +1,2 @@
+@import 'wobbl/variables';
+@import 'application';
--- a/app/javascript/styles/wobbl/variables.scss
+++ b/app/javascript/styles/wobbl/variables.scss
@ -0,0 +1,60 @@
+// Commonly used web colors
+$black: #000000;            // Black
+$white: #ffffff;            // White
+$success-green: #79bd9a !default;    // Padua
+$error-red: #df405a !default;        // Cerise
+$warning-red: #ff5050 !default;      // Sunset Orange
+$gold-star: #ca8f04 !default;        // Dark Goldenrod
+
+$red-bookmark: $warning-red;
+
+// Values from the classic Mastodon UI
+$classic-base-color: #191919;// Dark Gray
+$classic-primary-color: #E7E7E7;// Platinum
+$classic-secondary-color: #FF8680;    // Paler red
+$classic-highlight-color: #850700;    // Red highlights
+
+// Variables for defaults in UI
+$base-shadow-color: $black !default;
+$base-overlay-background: $black !default;
+$base-border-color: $white !default;
+$simple-background-color: $white !default;
+$valid-value-color: $success-green !default;
+$error-value-color: $error-red !default;
+
+// Tell UI to use selected colors
+$ui-base-color: $classic-base-color !default;                  // Darkest
+$ui-base-lighter-color: lighten($ui-base-color, 26%) !default; // Lighter darkest
+$ui-primary-color: $classic-primary-color !default;            // Lighter
+$ui-secondary-color: $classic-secondary-color !default;        // Lightest
+$ui-highlight-color: $classic-highlight-color !default;
+
+// Variables for texts
+$primary-text-color: $white !default;
+$darker-text-color: $ui-primary-color !default;
+$dark-text-color: $ui-base-lighter-color !default;
+$secondary-text-color: $ui-secondary-color !default;
+$highlight-text-color: lighten($ui-highlight-color, 8%) !default;
+$action-button-color: $ui-base-lighter-color !default;
+$passive-text-color: $gold-star !default;
+$active-passive-text-color: $success-green !default;
+
+// For texts on inverted backgrounds
+$inverted-text-color: $ui-base-color !default;
+$lighter-text-color: $ui-base-lighter-color !default;
+$light-text-color: $ui-primary-color !default;
+
+// Language codes that uses CJK fonts
+$cjk-langs: ja, ko, zh-CN, zh-HK, zh-TW;
+
+// Variables for components
+$media-modal-media-max-width: 100%;
+
+// put margins on top and bottom of image to avoid the screen covered by image.
+$media-modal-media-max-height: 80%;
+
+$no-gap-breakpoint: 1175px;
+
+$font-sans-serif: 'mastodon-font-sans-serif' !default;
+$font-display: 'mastodon-font-display' !default;
+$font-monospace: 'mastodon-font-monospace' !default;
--- a/app/lib/request.rb
+++ b/app/lib/request.rb
@ -7,11 +7,48 @@ require 'resolv'
 # Monkey-patch the HTTP.rb timeout class to avoid using a timeout block
 # around the Socket#open method, since we use our own timeout blocks inside
 # that method
+#
+# Also changes how the read timeout behaves so that it is cumulative (closer
+# to HTTP::Timeout::Global, but still having distinct timeouts for other
+# operation types)
 class HTTP::Timeout::PerOperation
  def connect(socket_class, host, port, nodelay = false)
    @socket = socket_class.open(host, port)
    @socket.setsockopt(Socket::IPPROTO_TCP, Socket::TCP_NODELAY, 1) if nodelay
  end
+
+  # Reset deadline when the connection is re-used for different requests
+  def reset_counter
+    @deadline = nil
+  end
+
+  # Read data from the socket
+  def readpartial(size, buffer = nil)
+    @deadline ||= Process.clock_gettime(Process::CLOCK_MONOTONIC) + @read_timeout
+
+    timeout = false
+    loop do
+      result = @socket.read_nonblock(size, buffer, exception: false)
+
+      return :eof if result.nil?
+
+      remaining_time = @deadline - Process.clock_gettime(Process::CLOCK_MONOTONIC)
+      raise HTTP::TimeoutError, "Read timed out after #{@read_timeout} seconds" if timeout || remaining_time <= 0
+      return result if result != :wait_readable
+
+      # marking the socket for timeout. Why is this not being raised immediately?
+      # it seems there is some race-condition on the network level between calling
+      # #read_nonblock and #wait_readable, in which #read_nonblock signalizes waiting
+      # for reads, and when waiting for x seconds, it returns nil suddenly without completing
+      # the x seconds. In a normal case this would be a timeout on wait/read, but it can
+      # also mean that the socket has been closed by the server. Therefore we "mark" the
+      # socket for timeout and try to read more bytes. If it returns :eof, it's all good, no
+      # timeout. Else, the first timeout was a proper timeout.
+      # This hack has to be done because io/wait#wait_readable doesn't provide a value for when
+      # the socket is closed by the server, and HTTP::Parser doesn't provide the limit for the chunks.
+      timeout = true unless @socket.to_io.wait_readable(remaining_time)
+    end
+  end
 end

 class Request
--- a/app/lib/scope_parser.rb
+++ b/app/lib/scope_parser.rb
@ -1,7 +1,7 @@
 # frozen_string_literal: true

 class ScopeParser < Parslet::Parser
-  rule(:term)      { match('[a-z]').repeat(1).as(:term) }
+  rule(:term)      { match('[a-z_]').repeat(1).as(:term) }
  rule(:colon)     { str(':') }
  rule(:access)    { (str('write') | str('read')).as(:access) }
  rule(:namespace) { str('admin').as(:namespace) }
--- a/app/lib/text_formatter.rb
+++ b/app/lib/text_formatter.rb
@ -48,6 +48,26 @@ class TextFormatter
    html.html_safe # rubocop:disable Rails/OutputSafety
  end

+  class << self
+    include ERB::Util
+
+    def shortened_link(url, rel_me: false)
+      url = Addressable::URI.parse(url).to_s
+      rel = rel_me ? (DEFAULT_REL + %w(me)) : DEFAULT_REL
+
+      prefix      = url.match(URL_PREFIX_REGEX).to_s
+      display_url = url[prefix.length, 30]
+      suffix      = url[prefix.length + 30..-1]
+      cutoff      = url[prefix.length..-1].length > 30
+
+      <<~HTML.squish
+        <a href="#{h(url)}" target="_blank" rel="#{rel.join(' ')}"><span class="invisible">#{h(prefix)}</span><span class="#{cutoff ? 'ellipsis' : ''}">#{h(display_url)}</span><span class="invisible">#{h(suffix)}</span></a>
+      HTML
+    rescue Addressable::URI::InvalidURIError, IDN::Idna::IdnaError
+      h(url)
+    end
+  end
+
  private

  def rewrite
@ -70,19 +90,7 @@ class TextFormatter
  end

  def link_to_url(entity)
-    url = Addressable::URI.parse(entity[:url]).to_s
-    rel = with_rel_me? ? (DEFAULT_REL + %w(me)) : DEFAULT_REL
-
-    prefix      = url.match(URL_PREFIX_REGEX).to_s
-    display_url = url[prefix.length, 30]
-    suffix      = url[prefix.length + 30..-1]
-    cutoff      = url[prefix.length..-1].length > 30
-
-    <<~HTML.squish
-      <a href="#{h(url)}" target="_blank" rel="#{rel.join(' ')}"><span class="invisible">#{h(prefix)}</span><span class="#{cutoff ? 'ellipsis' : ''}">#{h(display_url)}</span><span class="invisible">#{h(suffix)}</span></a>
-    HTML
-  rescue Addressable::URI::InvalidURIError, IDN::Idna::IdnaError
-    h(entity[:url])
+    TextFormatter.shortened_link(entity[:url], rel_me: with_rel_me?)
  end

  def link_to_hashtag(entity)
--- a/app/models/concerns/attachmentable.rb
+++ b/app/models/concerns/attachmentable.rb
@ -22,15 +22,14 @@ module Attachmentable

  included do
    def self.has_attached_file(name, options = {}) # rubocop:disable Naming/PredicateName
-      options = { validate_media_type: false }.merge(options)
      super(name, options)
-      send(:"before_#{name}_post_process") do
+
+      send(:"before_#{name}_validate") do
        attachment = send(name)
        check_image_dimension(attachment)
        set_file_content_type(attachment)
        obfuscate_file_name(attachment)
        set_file_extension(attachment)
-        Paperclip::Validators::MediaTypeSpoofDetectionValidator.new(attributes: [name]).validate(self)
      end
    end
  end
--- a/app/serializers/rest/instance_serializer.rb
+++ b/app/serializers/rest/instance_serializer.rb
@ -11,7 +11,7 @@ class REST::InstanceSerializer < ActiveModel::Serializer

  attributes :domain, :title, :version, :source_url, :description,
             :usage, :thumbnail, :languages, :configuration,
-             :registrations
+             :registrations, :max_toot_chars

  has_one :contact, serializer: ContactSerializer
  has_many :rules, serializer: REST::RuleSerializer
@ -88,6 +88,10 @@ class REST::InstanceSerializer < ActiveModel::Serializer
    }
  end

+  def max_toot_chars
+    1500
+  end
+
  private

  def registrations_enabled?
--- a/app/serializers/rest/preview_card_serializer.rb
+++ b/app/serializers/rest/preview_card_serializer.rb
@ -11,4 +11,8 @@ class REST::PreviewCardSerializer < ActiveModel::Serializer
  def image
    object.image? ? full_asset_url(object.image.url(:original)) : nil
  end
+
+  def html
+    Sanitize.fragment(object.html, Sanitize::Config::MASTODON_OEMBED)
+  end
 end
--- a/app/validators/status_length_validator.rb
+++ b/app/validators/status_length_validator.rb
@ -1,7 +1,7 @@
 # frozen_string_literal: true

 class StatusLengthValidator < ActiveModel::Validator
-  MAX_CHARS = 500
+  MAX_CHARS = 1500
  URL_PLACEHOLDER_CHARS = 23
  URL_PLACEHOLDER = 'x' * 23

--- a/config/application.rb
+++ b/config/application.rb
@ -28,6 +28,7 @@ require_relative '../lib/paperclip/url_generator_extensions'
 require_relative '../lib/paperclip/attachment_extensions'
 require_relative '../lib/paperclip/lazy_thumbnail'
 require_relative '../lib/paperclip/gif_transcoder'
+require_relative '../lib/paperclip/media_type_spoof_detector_extensions'
 require_relative '../lib/paperclip/transcoder'
 require_relative '../lib/paperclip/type_corrector'
 require_relative '../lib/paperclip/response_with_limit_adapter'
--- a/config/imagemagick/policy.xml
+++ b/config/imagemagick/policy.xml
@ -0,0 +1,27 @@
+<policymap>
+  <!-- Set some basic system resource limits -->
+  <policy domain="resource" name="time" value="60" />
+
+  <policy domain="module" rights="none" pattern="URL" />
+
+  <policy domain="filter" rights="none" pattern="*" />
+
+  <!--
+    Ideally, we would restrict ImageMagick to only accessing its own
+    disk-backed pixel cache as well as Mastodon-created Tempfiles.
+
+    However, those paths depend on the operating system and environment
+    variables, so they can only be known at runtime.
+
+    Furthermore, those paths are not necessarily shared across Mastodon
+    processes, so even creating a policy.xml at runtime is impractical.
+
+    For the time being, only disable indirect reads.
+  -->
+  <policy domain="path" rights="none" pattern="@*" />
+
+  <!-- Disallow any coder by default, and only enable ones required by Mastodon -->
+  <policy domain="coder" rights="none" pattern="*" />
+  <policy domain="coder" rights="read | write" pattern="{PNG,JPEG,GIF,HEIC,WEBP}" />
+  <policy domain="coder" rights="write" pattern="{HISTOGRAM,RGB,INFO}" />
+</policymap>
--- a/config/initializers/paperclip.rb
+++ b/config/initializers/paperclip.rb
@ -155,3 +155,10 @@ unless defined?(Seahorse)
    end
  end
 end
+
+# Set our ImageMagick security policy, but allow admins to override it
+ENV['MAGICK_CONFIGURE_PATH'] = begin
+  imagemagick_config_paths = ENV.fetch('MAGICK_CONFIGURE_PATH', '').split(File::PATH_SEPARATOR)
+  imagemagick_config_paths << Rails.root.join('config', 'imagemagick').expand_path.to_s
+  imagemagick_config_paths.join(File::PATH_SEPARATOR)
+end
--- a/config/locales/en.yml
+++ b/config/locales/en.yml
@ -1596,8 +1596,14 @@ en:
    does_not_match_previous_name: does not match the previous name
  themes:
    contrast: Mastodon (High contrast)
-    default: Mastodon (Dark)
+    dark: Mastodon (Dark)
    mastodon-light: Mastodon (Light)
+    wobbl: Mastodon (Dark - Wobbl theme)
+    wobbl-light: Mastodon (Light - Wobbl theme)
+    modern-dark: Modern (Dark)
+    modern-light: Modern (Light)
+    modern-light-wobbl: Modern (Light - Wobbl theme)
+    default: Modern (Dark - Wobbl theme) (Default)
  time:
    formats:
      default: "%b %d, %Y, %H:%M"
--- a/config/themes.yml
+++ b/config/themes.yml
@ -1,3 +1,9 @@
-default: styles/application.scss
+dark: styles/application.scss
 contrast: styles/contrast.scss
 mastodon-light: styles/mastodon-light.scss
+wobbl: styles/wobbl.scss
+wobbl-light: styles/wobbl-light.scss
+modern-dark: styles/modern-dark.scss
+modern-light: styles/modern-light.scss
+modern-light-wobbl: styles/modern-light-wobbl.scss
+default: styles/modern-dark-wobbl.scss
--- a/dist/nginx.conf
+++ b/dist/nginx.conf
@ -109,6 +109,8 @@ server {
  location ~ ^/system/ {
    add_header Cache-Control "public, max-age=2419200, immutable";
    add_header Strict-Transport-Security "max-age=63072000; includeSubDomains";
+    add_header X-Content-Type-Options nosniff;
+    add_header Content-Security-Policy "default-src 'none'; form-action 'none'";
    try_files $uri =404;
  }

--- a/lib/mastodon/version.rb
+++ b/lib/mastodon/version.rb
@ -13,7 +13,7 @@ module Mastodon
    end

    def patch
-      2
+      3
    end

    def flags
--- a/lib/paperclip/media_type_spoof_detector_extensions.rb
+++ b/lib/paperclip/media_type_spoof_detector_extensions.rb
@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+module Paperclip
+  module MediaTypeSpoofDetectorExtensions
+    def calculated_content_type
+      return @calculated_content_type if defined?(@calculated_content_type)
+
+      @calculated_content_type = type_from_file_command.chomp
+
+      # The `file` command fails to recognize some MP3 files as such
+      @calculated_content_type = type_from_marcel if @calculated_content_type == 'application/octet-stream' && type_from_marcel == 'audio/mpeg'
+      @calculated_content_type
+    end
+
+    def type_from_marcel
+      @type_from_marcel ||= Marcel::MimeType.for Pathname.new(@file.path),
+                                                 name: @file.path
+    end
+  end
+end
+
+Paperclip::MediaTypeSpoofDetector.prepend(Paperclip::MediaTypeSpoofDetectorExtensions)
--- a/lib/paperclip/transcoder.rb
+++ b/lib/paperclip/transcoder.rb
@ -19,10 +19,7 @@ module Paperclip
    def make
      metadata = VideoMetadataExtractor.new(@file.path)

-      unless metadata.valid?
-        Paperclip.log("Unsupported file #{@file.path}")
-        return File.open(@file.path)
-      end
+      raise Paperclip::Error, "Error while transcoding #{@file.path}: unsupported file" unless metadata.valid?

      update_attachment_type(metadata)
      update_options_from_metadata(metadata)
--- a/lib/public_file_server_middleware.rb
+++ b/lib/public_file_server_middleware.rb
@ -32,6 +32,11 @@ class PublicFileServerMiddleware
      end
    end

+    # Override the default CSP header set by the CSP middleware
+    headers['Content-Security-Policy'] = "default-src 'none'; form-action 'none'" if request_path.start_with?(paperclip_root_url)
+
+    headers['X-Content-Type-Options'] = 'nosniff'
+
    [status, headers, response]
  end

--- a/lib/sanitize_ext/sanitize_config.rb
+++ b/lib/sanitize_ext/sanitize_config.rb
@ -94,26 +94,26 @@ class Sanitize
      ]
    )

-    MASTODON_OEMBED ||= freeze_config merge(
-      RELAXED,
-      elements: RELAXED[:elements] + %w(audio embed iframe source video),
+    MASTODON_OEMBED ||= freeze_config(
+      elements: %w(audio embed iframe source video),

-      attributes: merge(
-        RELAXED[:attributes],
+      attributes: {
        'audio'  => %w(controls),
        'embed'  => %w(height src type width),
        'iframe' => %w(allowfullscreen frameborder height scrolling src width),
        'source' => %w(src type),
        'video'  => %w(controls height loop width),
-        'div'    => [:data]
-      ),
+      },

-      protocols: merge(
-        RELAXED[:protocols],
+      protocols: {
        'embed'  => { 'src' => HTTP_PROTOCOLS },
        'iframe' => { 'src' => HTTP_PROTOCOLS },
-        'source' => { 'src' => HTTP_PROTOCOLS }
-      )
+        'source' => { 'src' => HTTP_PROTOCOLS },
+      },
+
+      add_attributes: {
+        'iframe' => { 'sandbox' => 'allow-scripts allow-same-origin allow-popups allow-popups-to-escape-sandbox allow-forms' },
+      }
    )
  end
 end
--- a/spec/fixtures/files/boop.mp3
+++ b/spec/fixtures/files/boop.mp3
--- a/spec/models/media_attachment_spec.rb
+++ b/spec/models/media_attachment_spec.rb
@ -150,6 +150,26 @@ RSpec.describe MediaAttachment, type: :model do
    end
  end

+  describe 'mp3 with large cover art' do
+    let(:media) { described_class.create(account: Fabricate(:account), file: attachment_fixture('boop.mp3')) }
+
+    it 'detects it as an audio file' do
+      expect(media.type).to eq 'audio'
+    end
+
+    it 'sets meta for the duration' do
+      expect(media.file.meta['original']['duration']).to be_within(0.05).of(0.235102)
+    end
+
+    it 'extracts thumbnail' do
+      expect(media.thumbnail.present?).to be true
+    end
+
+    it 'gives the file a random name' do
+      expect(media.file_file_name).to_not eq 'boop.mp3'
+    end
+  end
+
  describe 'jpeg' do
    let(:media) { MediaAttachment.create(account: Fabricate(:account), file: attachment_fixture('attachment.jpg')) }

--- a/yarn.lock
+++ b/yarn.lock
@ -1461,6 +1461,17 @@
    "@types/yargs" "^17.0.8"
    chalk "^4.0.0"

+<<<<<<< Updated upstream
+=======
+"@jridgewell/gen-mapping@^0.1.0":
+  version "0.1.1"
+  resolved "https://registry.yarnpkg.com/@jridgewell/gen-mapping/-/gen-mapping-0.1.1.tgz#e5d2e450306a9491e3bd77e323e38d7aff315996"
+  integrity sha512-sQXCasFk+U8lWYEe66WxRDOE9PjVz4vSM51fTu3Hw+ClTpUSQb718772vH3pyS5pShp6lvQM7SxgIDXXXmOX7w==
+  dependencies:
+    "@jridgewell/set-array" "^1.0.0"
+    "@jridgewell/sourcemap-codec" "^1.4.10"
+
+>>>>>>> Stashed changes
 "@jridgewell/gen-mapping@^0.3.2":
  version "0.3.2"
  resolved "https://registry.yarnpkg.com/@jridgewell/gen-mapping/-/gen-mapping-0.3.2.tgz#c1aedc61e853f2bb9f5dfe6d4442d3b565b253b9"
Author	SHA1	Message	Date
Tarrien	1a7157000a	Theming and such	2023-07-06 13:23:31 -04:00
Tarrien	6c658d9680	Update to 4.1.3	2023-07-06 09:38:36 -04:00
Claire	0d5781ca76	Bump version to v4.1.3	2023-07-06 15:07:20 +02:00
Claire	32ebeed59b	Merge pull request from GHSA-55j9-c3mp-6fcq	2023-07-06 15:06:50 +02:00
Claire	e75ad1de0f	Merge pull request from GHSA-9pxv-6qvf-pjwc * Fix timeout handling of outbound HTTP requests * Use CLOCK_MONOTONIC instead of Time.now	2023-07-06 15:06:24 +02:00
Claire	0aa0b71f2c	Merge pull request from GHSA-9928-3cp5-93fm * Fix attachments getting processed despite failing content-type validation * Add a restrictive ImageMagick security policy tailored for Mastodon * Fix misdetection of MP3 files with large cover art * Reject unprocessable audio/video files instead of keeping them unchanged	2023-07-06 15:05:05 +02:00
Claire	c4f2609f7a	Merge pull request from GHSA-ccm4-vgcc-73hp * Tighten allowed HTML in oEmbed-based preview cards * Sanitize preview cards at render time * Add `sandbox` attribute to preview card iframes	2023-07-06 15:03:33 +02:00
Claire	9b6c0cac7d	Add hardened headers to user-uploaded files (#25756 )	2023-07-06 14:32:26 +02:00
Claire	fac2c9eb7d	Update rack, rails, nokogiri and doorkeeper gems	2023-07-06 13:45:40 +02:00
Claire	a3d69a2c5d	Fix OAuth apps page crashing when listing apps with certain admin API scopes (#25713 )	2023-07-06 13:45:40 +02:00
				`@ -0,0 +1 @@`
				`Subproject commit 301d0033f593a35a289a22f1891a0dfe12245d7f`