Merge #2961

2961: Hardened malloc was not disabled for oletools when an CPU with missing flags is used r=Diman0 a=Diman0

## What type of PR?
bug fix

## What does this PR do?
Updates oletools to also disable hardened malloc when used CPU misses flags

### Related issue(s)
- closes #2959 

## Prerequisites
Before we can consider review and merge, please make sure the following list is done and checked.
If an entry in not applicable, you can check it or remove it from the list.

- [n/a ] In case of feature or enhancement: documentation updated accordingly
- [x] Unless it's docs or a minor change: add [changelog](https://mailu.io/master/contributors/workflow.html#changelog) entry file.


Co-authored-by: Dimitri Huisman <diman@huisman.xyz>
Co-authored-by: Dimitri Huisman <52963853+Diman0@users.noreply.github.com>
Co-authored-by: Florent Daigniere <nextgens@users.noreply.github.com>
Co-authored-by: Florent Daigniere <nextgens@freenetproject.org>

core/base/Dockerfile

@@ -79,7 +79,6 @@ COPY --chown=root:root --from=build /app/snuffleupagus.so /usr/lib/php81/modules
 ENV \
   VIRTUAL_ENV=/app/venv \
   PATH="/app/venv/bin:${PATH}" \
-  LD_PRELOAD="/usr/lib/libhardened_malloc.so" \
   ADMIN_ADDRESS="admin" \
   FRONT_ADDRESS="front" \
   SMTP_ADDRESS="smtp" \

core/base/libs/socrate/socrate/system.py

@@ -66,7 +66,8 @@ def _is_compatible_with_hardened_malloc():
         lines = f.readlines()
         for line in lines:
             # See #2764, we need vmovdqu
-            if line.startswith('flags') and ' avx ' not in line:
+            # See #2959, we need vpunpckldq
+            if line.startswith('flags') and ' avx2 ' not in line:
                 return False
             # See #2541
             if line.startswith('Features') and ' lrcpc ' not in line:
@@ -79,9 +80,9 @@ def set_env(required_secrets=[], log_filters=[], log_file=None):
         sys.stderr = LogFilter(sys.stderr, log_filters, log_file)
     log.basicConfig(stream=sys.stderr, level=os.environ.get("LOG_LEVEL", 'WARNING'))
 
-    if 'LD_PRELOAD' in os.environ and not _is_compatible_with_hardened_malloc():
-        log.warning('Disabling hardened-malloc on this CPU')
-        del os.environ['LD_PRELOAD']
+    if not 'LD_PRELOAD' in os.environ and _is_compatible_with_hardened_malloc():
+        log.warning('Your CPU has Advanced Vector Extensions available, we recommend you enable hardened-malloc earlier in the boot process by adding LD_PRELOAD=/usr/lib/libhardened_malloc.so to your mailu.env')
+        os.environ['LD_PRELOAD'] = '/usr/lib/libhardened_malloc.so'
 
     """ This will set all the environment variables and retains only the secrets we need """
     if 'SECRET_KEY_FILE' in os.environ:

core/oletools/Dockerfile

@@ -6,11 +6,17 @@ FROM base
 ARG VERSION=local
 LABEL version=$VERSION
 
+ARG OLEFY_SCRIPT=https://raw.githubusercontent.com/HeinleinSupport/olefy/f8aac6cc55283886d153e89c8f27fae66b1c24e2/olefy.py
+ARG OLEFY_SHA256=1f5aa58b78ca7917350135b4425e5ed4d580c7051aabed1952c6afd12d0345a0
+
 RUN set -euxo pipefail \
   ; apk add --no-cache netcat-openbsd libmagic libffi \
-  ; curl -sLo olefy.py https://raw.githubusercontent.com/HeinleinSupport/olefy/f8aac6cc55283886d153e89c8f27fae66b1c24e2/olefy.py \
+  ; curl -sLo olefy.py $OLEFY_SCRIPT \
+  ; echo "$OLEFY_SHA256  olefy.py" |sha256sum -c \
   ; chmod 755 olefy.py
 
+COPY start.py /
+
 RUN echo $VERSION >/version
 
 HEALTHCHECK --start-period=60s CMD echo PING|nc -q1 127.0.0.1 11343|grep "PONG"
@@ -28,4 +34,4 @@ ENV \
     OLEFY_DEL_TMP="1" \
     OLEFY_DEL_TMP_FAILED="1"
 
-CMD /app/olefy.py
+CMD /start.py

core/oletools/start.py

@@ -0,0 +1,8 @@
+#!/usr/bin/env python3
+
+from socrate import system
+
+system.set_env()
+
+with open('/app/olefy.py') as olefy:
+    exec(olefy.read())

docs/compose/.env

@@ -152,3 +152,8 @@ REJECT_UNLISTED_RECIPIENT=
 
 # Log level threshold in start.py (value: CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET)
 LOG_LEVEL=WARNING
+
+# If your CPU supports Advanced Vector Extensions
+# (AVX2 on x86_64, lrcpc on ARM64), you should consider enabling
+# hardened-malloc earlier by uncommenting this
+# LD_PRELOAD=/usr/lib/libhardened_malloc.so

docs/compose/setup.rst

@@ -76,6 +76,15 @@ Review configuration variables
 After downloading the files, open ``mailu.env`` and review the variable settings.
 Make sure to read the comments in the file and instructions from the :ref:`common_cfg` page.
 
+If your CPU supports Advanced Vector Extensions (AVX2 on x86_64, lrcpc on ARM64), you should
+consider enabling hardened-malloc earlier in the boot process by adding the following to
+your mailu.env:
+
+.. code-block:: bash
+
+    LD_PRELOAD=/usr/lib/libhardened_malloc.so
+
+
 Finish setting up TLS
 ---------------------
 

towncrier/newsfragments/2959.bugfix

@@ -0,0 +1,3 @@
+Update hardened malloc as the original package is not available from alpine anymore.
+The newer version of hardened malloc requires AVX2: Disable it by default at startup and hint in the logs when it should be enabled instead.
+Upgrade snappymail to v2.29.1

webmails/Dockerfile

@@ -52,7 +52,7 @@ COPY roundcube/config/config.inc.carddav.php /var/www/roundcube/plugins/carddav/
 
 # snappymail
 
-ENV SNAPPYMAIL_URL https://github.com/the-djmaze/snappymail/releases/download/v2.28.4/snappymail-2.28.4.tar.gz
+ENV SNAPPYMAIL_URL https://github.com/the-djmaze/snappymail/releases/download/v2.29.1/snappymail-2.29.1.tar.gz
 
 RUN set -euxo pipefail \
   ; mkdir  /var/www/snappymail \