go/mailpit
1
0
Fork 0
mirror of https://github.com/axllent/mailpit.git synced 2025-02-05 13:14:57 +02:00
Code Issues Releases Activity

Merge branch 'feature/check-links' into develop

Browse Source
This commit is contained in:
Ralph Slooten 2023-08-16 17:03:02 +12:00
commit 72d780fe66
19 changed files with 869 additions and 44 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
README.md
View File

@ -6,11 +6,11 @@
![CodeQL](https://github.com/axllent/mailpit/actions/workflows/codeql-analysis.yml/badge.svg)
[![Go Report Card](https://goreportcard.com/badge/github.com/axllent/mailpit)](https://goreportcard.com/report/github.com/axllent/mailpit)
Mailpit is a multi-platform email testing tool & API for developers.
Mailpit is a small, fast, low memory, zero-dependency, multi-platform email testing tool & API for developers.
It acts as both an SMTP server, and provides a web interface to view all captured emails. It also contains an API for automated integration testing.
It acts as an SMTP server, provides a modern web interface to view & test captured emails, and contains an API for automated integration testing.
Mailpit is inspired by [MailHog](#why-rewrite-mailhog), but modern and much, much faster.
Mailpit was originally **inspired** by MailHog which is now [no longer maintained](https://github.com/mailhog/MailHog/issues/442#issuecomment-1493415258) and hasn't seen active development for a few years now.
![Mailpit](https://raw.githubusercontent.com/axllent/mailpit/develop/docs/screenshot.png)
@ -21,6 +21,7 @@ Mailpit is inspired by [MailHog](#why-rewrite-mailhog), but modern and much, muc
- SMTP server (default `0.0.0.0:1025`)
- Web UI to view emails (formatted HTML, highlighted HTML source, text, headers, raw source and MIME attachments including image thumbnails)
- HTML check to test & score mail client compatibility with HTML emails
- Link check to test message links (HTML & text) & linked images
- Light & dark web UI theme with auto-detect
- Mobile and tablet HTML preview toggle in desktop mode
- Advanced mail search ([see wiki](https://github.com/axllent/mailpit/wiki/Mail-search))
@ -88,12 +89,3 @@ Please refer to [the documentation](https://github.com/axllent/mailpit/wiki/Test
### Configuring sendmail
Mailpit's SMTP server (by default on port 1025), so you will likely need to configure your sending application to deliver mail via that port. A common MTA (Mail Transfer Agent) that delivers system emails to a SMTP server is `sendmail`, used by many applications including PHP. Mailpit can also act as substitute for sendmail. For instructions of how to set this up, please refer to the [sendmail documentation](https://github.com/axllent/mailpit/wiki/Configuring-sendmail).
## Why rewrite MailHog?
I had been using MailHog for a few years to intercept and test emails, but experienced a number of severe performance issues. Many of the frontend and Go libraries are very out of date, and the project [is no longer maintained](https://github.com/mailhog/MailHog/issues/442#issuecomment-1493415258).
Initially I tried to upgrade a fork of MailHog (the UI, the HTTP server and the API), but discovered that it is (with all due respect to its authors) far too complex. I found it over-engineered (split over 9 separate projects), and performs very poorly when dealing with large amounts of emails or emails with attachments (a single email with a 3MB attachment can take over a minute to ingest). Finally the API transmits a lot of duplicate & irrelevant data on every browser request, all without any HTTP compression.
In order to improve it I felt it needed to be completely rewritten, and so Mailpit was born.

4
cmd/root.go
View File

@ -100,6 +100,7 @@ func init() {
rootCmd.Flags().StringVar(&config.SMTPTLSCert, "smtp-tls-cert", config.SMTPTLSCert, "TLS certificate for SMTP (STARTTLS) - requires smtp-tls-key")
rootCmd.Flags().StringVar(&config.SMTPTLSKey, "smtp-tls-key", config.SMTPTLSKey, "TLS key for SMTP (STARTTLS) - requires smtp-tls-cert")
rootCmd.Flags().BoolVar(&config.SMTPAuthAllowInsecure, "smtp-auth-allow-insecure", config.SMTPAuthAllowInsecure, "Enable insecure PLAIN & LOGIN authentication")
rootCmd.Flags().BoolVar(&config.SMTPStrictRFCHeaders, "smtp-strict-rfc-headers", config.SMTPStrictRFCHeaders, "Return SMTP error if message headers contain <CR><CR><LF>")
rootCmd.Flags().StringVar(&config.SMTPRelayConfigFile, "smtp-relay-config", config.SMTPRelayConfigFile, "SMTP configuration file to allow releasing messages")
rootCmd.Flags().BoolVar(&config.SMTPRelayAllIncoming, "smtp-relay-all", config.SMTPRelayAllIncoming, "Relay all incoming messages via external SMTP server (caution!)")
@ -185,6 +186,9 @@ func initConfigFromEnv() {
if getEnabledFromEnv("MP_SMTP_AUTH_ALLOW_INSECURE") {
config.SMTPAuthAllowInsecure = true
}
if getEnabledFromEnv("MP_STRICT_RFC_HEADERS") {
config.SMTPStrictRFCHeaders = true
}
// Relay server config
if len(os.Getenv("MP_SMTP_RELAY_CONFIG")) > 0 {

4
config/config.go
View File

@ -90,6 +90,10 @@ var (
// SMTPRelayConfig to parse a yaml file and store config of relay SMTP server
SMTPRelayConfig smtpRelayConfigStruct
// SMTPStrictRFCHeaders will return an error if the email headers contain <CR><CR><LF> (\r\r\n)
// @see https://github.com/axllent/mailpit/issues/87 & https://github.com/axllent/mailpit/issues/153
SMTPStrictRFCHeaders bool
// ReleaseEnabled is whether message releases are enabled, requires a valid SMTPRelayConfigFile
ReleaseEnabled = false

59
server/apiv1/api.go
View File

@ -14,6 +14,7 @@ import (
"github.com/axllent/mailpit/server/smtpd"
"github.com/axllent/mailpit/storage"
"github.com/axllent/mailpit/utils/htmlcheck"
"github.com/axllent/mailpit/utils/linkcheck"
"github.com/axllent/mailpit/utils/logger"
"github.com/axllent/mailpit/utils/tools"
"github.com/gorilla/mux"
@ -638,7 +639,7 @@ func HTMLCheck(w http.ResponseWriter, r *http.Request) {
//
// # HTML check (beta)
//
// Returns the summary of HTML check.
// Returns the summary of the message HTML checker.
//
// NOTE: This feature is currently in beta and is documented for reference only.
// Please do not integrate with it (yet) as there may be changes.
@ -684,6 +685,62 @@ func HTMLCheck(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write(bytes)
}
// LinkCheck returns a summary of links in the email
func LinkCheck(w http.ResponseWriter, r *http.Request) {
// swagger:route GET /api/v1/message/{ID}/link-check Other LinkCheckResponse
//
// # Link check (beta)
//
// Returns the summary of the message Link checker.
//
// NOTE: This feature is currently in beta and is documented for reference only.
// Please do not integrate with it (yet) as there may be changes.
//
// Produces:
// - application/json
//
// Schemes: http, https
//
// Parameters:
// + name: ID
// in: path
// description: Database ID
// required: true
// type: string
// + name: follow
// in: query
// description: Follow redirects
// required: false
// type: boolean
// default: false
//
// Responses:
// 200: LinkCheckResponse
// default: ErrorResponse
vars := mux.Vars(r)
id := vars["id"]
msg, err := storage.GetMessage(id)
if err != nil {
fourOFour(w)
return
}
f := r.URL.Query().Get("follow")
followRedirects := f == "true" || f == "1"
summary, err := linkcheck.RunTests(msg, followRedirects)
if err != nil {
httpError(w, err.Error())
return
}
bytes, _ := json.Marshal(summary)
w.Header().Add("Content-Type", "application/json")
_, _ = w.Write(bytes)
}
// FourOFour returns a basic 404 message
func fourOFour(w http.ResponseWriter) {
w.Header().Set("Referrer-Policy", "no-referrer")

4
server/apiv1/structs.go
View File

@ -3,6 +3,7 @@ package apiv1
import (
"github.com/axllent/mailpit/storage"
"github.com/axllent/mailpit/utils/htmlcheck"
"github.com/axllent/mailpit/utils/linkcheck"
)
// MessagesSummary is a summary of a list of messages
@ -49,3 +50,6 @@ type Attachment = storage.Attachment
// HTMLCheckResponse summary
type HTMLCheckResponse = htmlcheck.Response
// LinkCheckResponse summary
type LinkCheckResponse = linkcheck.Response

1
server/server.go
View File

@ -95,6 +95,7 @@ func defaultRoutes() *mux.Router {
if !config.DisableHTMLCheck {
r.HandleFunc(config.Webroot+"api/v1/message/{id}/html-check", middleWareFunc(apiv1.HTMLCheck)).Methods("GET")
}
r.HandleFunc(config.Webroot+"api/v1/message/{id}/link-check", middleWareFunc(apiv1.LinkCheck)).Methods("GET")
r.HandleFunc(config.Webroot+"api/v1/message/{id}", middleWareFunc(apiv1.GetMessage)).Methods("GET")
r.HandleFunc(config.Webroot+"api/v1/info", middleWareFunc(apiv1.AppInfo)).Methods("GET")
r.HandleFunc(config.Webroot+"api/v1/webui", middleWareFunc(apiv1.WebUIConfig)).Methods("GET")

6
server/smtpd/smtpd.go
View File

@ -17,6 +17,12 @@ import (
)
func mailHandler(origin net.Addr, from string, to []string, data []byte) error {
if !config.SMTPStrictRFCHeaders {
// replace all <CR><CR><LF> (\r\r\n) with <CR><LF> (\r\n)
// @see https://github.com/axllent/mailpit/issues/87 & https://github.com/axllent/mailpit/issues/153
data = bytes.ReplaceAll(data, []byte("\r\r\n"), []byte("\r\n"))
}
msg, err := mail.ReadMessage(bytes.NewReader(data))
if err != nil {
logger.Log().Errorf("[smtpd] error parsing message: %s", err.Error())

6
server/ui-src/App.vue
View File

@ -88,6 +88,10 @@ export default {
},
mounted() {
let title = document.title + ' - ' + location.hostname
document.title = title
this.currentPath = window.location.hash.slice(1)
window.addEventListener('hashchange', () => {
this.currentPath = window.location.hash.slice(1)
@ -1006,7 +1010,7 @@ export default {
</div>
</div>
<Message v-if="message" :message="message" :existingTags="existingTags" :uiConfig="uiConfig"
<Message v-if="message" :message="message" :existingTags="existingTags" :uiConfig="uiConfig" :key="message.ID"
@load-messages="loadMessages">
</Message>
</div>

5
server/ui-src/assets/styles.scss
View File

@ -39,14 +39,13 @@
}
.nav-tabs .nav-link {
@include media-breakpoint-down(sm) {
// font-size: 14px;
@include media-breakpoint-down(xl) {
padding-left: 10px;
padding-right: 10px;
}
}
:not(.text-view) > a {
:not(.text-view) > a:not(.no-icon) {
&[href^="http://"],
&[href^="https://"]
{

67
server/ui-src/templates/Message.vue
View File

@ -6,6 +6,7 @@ import Tags from "bootstrap5-tags"
import Attachments from './Attachments.vue'
import Headers from './Headers.vue'
import HTMLCheck from './MessageHTMLCheck.vue'
import LinkCheck from './MessageLinkCheck.vue'
export default {
props: {
@ -18,6 +19,7 @@ export default {
Attachments,
Headers,
HTMLCheck,
LinkCheck,
},
mixins: [commonMixins],
@ -33,6 +35,7 @@ export default {
loadHeaders: false,
htmlScore: false,
htmlScoreColor: false,
linkCheckErrors: false,
showMobileButtons: false,
scaleHTMLPreview: 'display',
// keys names match bootstrap icon names
@ -219,7 +222,7 @@ export default {
let html = s
// full links with http(s)
let re = /(\b(https?|ftp):\/\/[\-\w@:%_\+.~#?,&\/\/=;]+)\b/gim
let re = /(\b(https?|ftp):\/\/[\-\w@:%_\+'!.~#?,&\/\/=;]+)/gim
html = html.replace(re, '˱˱˱a href=ˠˠˠ$&ˠˠˠ target=_blank rel=noopener˲˲˲$&˱˱˱/a˲˲˲')
// plain www links without https?:// prefix
@ -366,15 +369,51 @@ export default {
role="tab" aria-controls="nav-raw" aria-selected="false">
Raw
</button>
<button class="nav-link position-relative" id="nav-html-check-tab" data-bs-toggle="tab"
data-bs-target="#nav-html-check" type="button" role="tab" aria-controls="nav-html" aria-selected="false"
v-if="!uiConfig.DisableHTMLCheck && message.HTML != ''" @click="showMobileButtons = false">
<span class="d-none d-sm-inline">HTML</span> Check
<span class="position-absolute top-10 start-100 translate-middle badge rounded-pill p-1"
:class="htmlScoreColor" v-if="htmlScore !== false">
<div class="dropdown d-lg-none">
<button class="nav-link dropdown-toggle" type="button" data-bs-toggle="dropdown" aria-expanded="false">
Checks
</button>
<ul class="dropdown-menu">
<li>
<button class="dropdown-item" id="nav-html-check-tab" data-bs-toggle="tab"
data-bs-target="#nav-html-check" type="button" role="tab" aria-controls="nav-html"
aria-selected="false" v-if="!uiConfig.DisableHTMLCheck && message.HTML != ''">
HTML Check
<span class="badge rounded-pill p-1" :class="htmlScoreColor" v-if="htmlScore !== false">
<small>{{ Math.floor(htmlScore) }}%</small>
</span>
</button>
</li>
<li>
<button class="dropdown-item" id="nav-link-check-tab" data-bs-toggle="tab"
data-bs-target="#nav-link-check" type="button" role="tab" aria-controls="nav-link-check"
aria-selected="false">
Link Check
<i class="bi bi-check-all text-success" v-if="linkCheckErrors === 0"></i>
<span class="badge rounded-pill bg-danger" v-else-if="linkCheckErrors > 0">
<small>{{ formatNumber(linkCheckErrors) }}</small>
</span>
</button>
</li>
</ul>
</div>
<button class="d-none d-lg-inline-block nav-link position-relative" id="nav-html-check-tab"
data-bs-toggle="tab" data-bs-target="#nav-html-check" type="button" role="tab" aria-controls="nav-html"
aria-selected="false" v-if="!uiConfig.DisableHTMLCheck && message.HTML != ''">
HTML Check
<span class="badge rounded-pill p-1" :class="htmlScoreColor" v-if="htmlScore !== false">
<small>{{ Math.floor(htmlScore) }}%</small>
</span>
</button>
<button class="d-none d-lg-inline-block nav-link" id="nav-link-check-tab" data-bs-toggle="tab"
data-bs-target="#nav-link-check" type="button" role="tab" aria-controls="nav-link-check"
aria-selected="false">
Link Check
<i class="bi bi-check-all text-success" v-if="linkCheckErrors === 0"></i>
<span class="badge rounded-pill bg-danger" v-else-if="linkCheckErrors > 0">
<small>{{ formatNumber(linkCheckErrors) }}</small>
</span>
</button>
<div class="d-none d-lg-block ms-auto me-3" v-if="showMobileButtons">
<template v-for="vals, key in responsiveSizes">
@ -398,11 +437,6 @@ export default {
<Attachments v-if="allAttachments(message).length" :message="message"
:attachments="allAttachments(message)"></Attachments>
</div>
<div class="tab-pane fade" id="nav-html-check" role="tabpanel" aria-labelledby="nav-html-check-tab"
tabindex="0">
<HTMLCheck v-if="!uiConfig.DisableHTMLCheck && message.HTML != ''" :message="message"
@setHtmlScore="(n) => htmlScore = n" @set-badge-style="(v) => htmlScoreColor = v" />
</div>
<div class="tab-pane fade" id="nav-html-source" role="tabpanel" aria-labelledby="nav-html-source-tab"
tabindex="0" v-if="message.HTML">
<pre><code class="language-html">{{ message.HTML }}</code></pre>
@ -420,6 +454,15 @@ export default {
<iframe v-if="srcURI" :src="srcURI" v-on:load="initRawIframe" frameborder="0"
style="width: 100%; height: 300px"></iframe>
</div>
<div class="tab-pane fade" id="nav-html-check" role="tabpanel" aria-labelledby="nav-html-check-tab"
tabindex="0">
<HTMLCheck v-if="!uiConfig.DisableHTMLCheck && message.HTML != ''" :message="message"
@setHtmlScore="(n) => htmlScore = n" @set-badge-style="(v) => htmlScoreColor = v" />
</div>
<div class="tab-pane fade" id="nav-link-check" role="tabpanel" aria-labelledby="nav-html-check-tab"
tabindex="0">
<LinkCheck :message="message" @setLinkErrors="(n) => linkCheckErrors = n" />
</div>
</div>
</div>
</template>

2
server/ui-src/templates/MessageHTMLCheck.vue
View File

@ -640,7 +640,7 @@ export default {
<div class="modal-dialog modal-lg modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<h1 class="modal-title fs-5" id="HTMLCheckOptionsLabel">About HTML check</h1>
<h1 class="modal-title fs-5" id="HTMLCheckOptionsLabel">HTML check options</h1>
<button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
</div>
<div class="modal-body">

398
server/ui-src/templates/MessageLinkCheck.vue Normal file
View File

@ -0,0 +1,398 @@
<script>
import axios from 'axios'
import commonMixins from '../mixins.js'
export default {
props: {
message: Object,
},
emits: ["setLinkErrors"],
mixins: [commonMixins],
data() {
return {
error: false,
autoScan: false,
followRedirects: false,
check: false,
loaded: false,
loading: false,
}
},
created() {
this.autoScan = localStorage.getItem('LinkCheckAutoScan')
this.followRedirects = localStorage.getItem('LinkCheckFollowRedirects')
},
mounted() {
this.loaded = true
if (this.autoScan) {
this.doCheck()
}
},
watch: {
autoScan(v) {
if (!this.loaded) {
return
}
if (v) {
localStorage.setItem('LinkCheckAutoScan', true)
if (!this.check) {
this.doCheck()
}
} else {
localStorage.removeItem('LinkCheckAutoScan')
}
},
followRedirects(v) {
if (!this.loaded) {
return
}
if (v) {
localStorage.setItem('LinkCheckFollowRedirects', true)
} else {
localStorage.removeItem('LinkCheckFollowRedirects')
}
if (this.check) {
this.doCheck()
}
}
},
computed: {
groupedStatuses: function () {
let results = {}
if (!this.check) {
return results
}
// group by status
this.check.Links.forEach(function (r) {
if (!results[r.StatusCode]) {
let css = ""
if (r.StatusCode >= 400 || r.StatusCode === 0) {
css = "text-danger"
} else if (r.StatusCode >= 300) {
css = "text-info"
}
if (r.StatusCode === 0) {
r.Status = 'Cannot connect to server'
}
results[r.StatusCode] = {
StatusCode: r.StatusCode,
Status: r.Status,
Class: css,
URLS: []
}
}
results[r.StatusCode].URLS.push(r.URL)
})
let newArr = []
for (const i in results) {
newArr.push(results[i])
}
// sort statuses
let sorted = newArr.sort((a, b) => {
if (a.StatusCode === 0) {
return false
}
return a.StatusCode < b.StatusCode
})
return sorted
}
},
methods: {
doCheck: function () {
this.check = false
let self = this
this.loading = true
let uri = 'api/v1/message/' + self.message.ID + '/link-check'
if (this.followRedirects) {
uri += '?follow=true'
}
// ignore any error, do not show loader
axios.get(uri, null)
.then(function (result) {
self.check = result.data
self.error = false
self.$emit('setLinkErrors', result.data.Errors)
})
.catch(function (error) {
// handle error
if (error.response && error.response.data) {
// The request was made and the server responded with a status code
// that falls out of the range of 2xx
if (error.response.data.Error) {
self.error = error.response.data.Error
} else {
self.error = error.response.data
}
} else if (error.request) {
// The request was made but no response was received
// `error.request` is an instance of XMLHttpRequest in the browser and an instance of
// http.ClientRequest in node.js
self.error = 'Error sending data to the server. Please try again.'
} else {
// Something happened in setting up the request that triggered an Error
self.error = error.message
}
})
.then(function (result) {
// always run
self.loading = false
})
},
}
}
</script>
<template>
<div class="pe-3">
<div class="row mb-3 align-items-center">
<div class="col">
<h4 class="mb-0">
<template v-if="!check">
Link check
</template>
<template v-else>
<template v-if="check.Links.length">
Scanned {{ formatNumber(check.Links.length) }}
link<template v-if="check.Links.length != 1">s</template>
</template>
<template v-else>
No links detected
</template>
</template>
</h4>
</div>
<div class="col-auto">
<div class="input-group">
<button class="btn btn-outline-secondary" data-bs-toggle="modal"
data-bs-target="#AboutLinkCheckResults">
<i class="bi bi-info-circle-fill"></i>
Help
</button>
<button class="btn btn-outline-secondary" data-bs-toggle="modal" data-bs-target="#LinkCheckOptions">
<i class="bi bi-gear-fill"></i>
Settings
</button>
</div>
</div>
</div>
<div v-if="!check">
<p class="text-secondary">
Link check scans your email text &amp; HTML for unique links, testing the response status codes.
This includes links to images and remote CSS stylesheets.
</p>
<p class="text-center my-5">
<button v-if="!check" class="btn btn-primary btn-lg" @click="doCheck()" :disabled="loading">
<template v-if="loading">
Checking links
<div class="ms-1 spinner-border spinner-border-sm text-light" role="status">
<span class="visually-hidden">Loading...</span>
</div>
</template>
<template v-else>
<i class="bi bi-check-square me-2"></i>
Check message links
</template>
</button>
</p>
</div>
<div v-else v-for="s, k in groupedStatuses">
<div class="card mb-3">
<div class="card-header h4" :class="s.Class">
Status {{ s.StatusCode }}
<small v-if="s.Status != ''" class="ms-2 small text-secondary">({{ s.Status }})</small>
</div>
<ul class="list-group list-group-flush">
<li v-for="u in s.URLS" class="list-group-item">
<a :href="u" target="_blank" class="no-icon">{{ u }}</a>
</li>
</ul>
</div>
</div>
<template v-if="error">
<p>Link check failed to load:</p>
<div class="alert alert-warning">
{{ error }}
</div>
</template>
</div>
<div class="modal fade" id="LinkCheckOptions" tabindex="-1" aria-labelledby="LinkCheckOptionsLabel" aria-hidden="true">
<div class="modal-dialog modal-lg modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<h1 class="modal-title fs-5" id="LinkCheckOptionsLabel">Link check options</h1>
<button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
</div>
<div class="modal-body">
<p>
Link check is currently in beta. Constructive feedback is welcome via
<a href="https://github.com/axllent/mailpit/issues" target="_blank">GitHub</a>.
</p>
<h6 class="mt-4">Follow HTTP redirects (status 301 & 302)</h6>
<div class="form-check form-switch mb-4">
<input class="form-check-input" type="checkbox" role="switch" v-model="followRedirects"
id="LinkCheckFollowRedirectsSwitch">
<label class="form-check-label" for="LinkCheckFollowRedirectsSwitch">
<template v-if="followRedirects">Following HTTP redirects</template>
<template v-else>Not following HTTP redirects</template>
</label>
</div>
<h6 class="mt-4">Automatic link checking</h6>
<div class="form-check form-switch mb-3">
<input class="form-check-input" type="checkbox" role="switch" v-model="autoScan"
id="LinkCheckAutoCheckSwitch">
<label class="form-check-label" for="LinkCheckAutoCheckSwitch">
<template v-if="autoScan">Automatic link checking is enabled</template>
<template v-else>Automatic link checking is disabled</template>
</label>
<div class="form-text">
Note: Enabling auto checking will scan every link & image every time a message is opened.
Only enable this if you understand the potential risks &amp; consequences.
</div>
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Close</button>
</div>
</div>
</div>
</div>
<div class="modal fade" id="AboutLinkCheckResults" tabindex="-1" aria-labelledby="AboutLinkCheckResultsLabel"
aria-hidden="true">
<div class="modal-dialog modal-lg modal-dialog-scrollable">
<div class="modal-content">
<div class="modal-header">
<h1 class="modal-title fs-5" id="AboutLinkCheckResultsLabel">About Link check</h1>
<button type="button" class="btn-close" data-bs-dismiss="modal" aria-label="Close"></button>
</div>
<div class="modal-body">
<p>
Link check is currently in beta. Constructive feedback is welcome via
<a href="https://github.com/axllent/mailpit/issues" target="_blank">GitHub</a>.
</p>
<div class="accordion" id="LinkCheckAboutAccordion">
<div class="accordion-item">
<h2 class="accordion-header">
<button class="accordion-button collapsed" type="button" data-bs-toggle="collapse"
data-bs-target="#col1" aria-expanded="false" aria-controls="col1">
What is Link check?
</button>
</h2>
<div id="col1" class="accordion-collapse collapse" data-bs-parent="#LinkCheckAboutAccordion">
<div class="accordion-body">
Link check scans your message HTML and text for all unique links, images and linked
stylesheets. It then does a HTTP <code>HEAD</code> request to each link, 5 at a time, to
test whether the link/image/stylesheet exists.
</div>
</div>
</div>
<div class="accordion-item">
<h2 class="accordion-header">
<button class="accordion-button collapsed" type="button" data-bs-toggle="collapse"
data-bs-target="#col2" aria-expanded="false" aria-controls="col2">
What are "301" and "302" links?
</button>
</h2>
<div id="col2" class="accordion-collapse collapse" data-bs-parent="#LinkCheckAboutAccordion">
<div class="accordion-body">
<p>
These are links that redirect you to another URL, for example newsletters
often use redirect links to track user clicks.
</p>
<p>
By default Link check will not follow these links, however you can turn this on via
the settings and Link check will "follow" those redirects.
</p>
</div>
</div>
</div>
<div class="accordion-item">
<h2 class="accordion-header">
<button class="accordion-button collapsed" type="button" data-bs-toggle="collapse"
data-bs-target="#col3" aria-expanded="false" aria-controls="col3">
Why are some links returning an error but work in my browser?
</button>
</h2>
<div id="col3" class="accordion-collapse collapse" data-bs-parent="#LinkCheckAboutAccordion">
<div class="accordion-body">
<p>This may be due to various reasons, for instance:</p>
<ul>
<li>The Mailpit server cannot resolve (DNS) the hostname of the URL.</li>
<li>Mailpit is not allowed to access the URL.</li>
<li>
The webserver is blocking requests that don't come from authenticated web
browsers.
</li>
<li>The webserver or doesn't allow HTTP <code>HEAD</code> requests. </li>
</ul>
</div>
</div>
</div>
<div class="accordion-item">
<h2 class="accordion-header">
<button class="accordion-button collapsed" type="button" data-bs-toggle="collapse"
data-bs-target="#col4" aria-expanded="false" aria-controls="col4">
What are the risks of running Link check automatically?
</button>
</h2>
<div id="col4" class="accordion-collapse collapse" data-bs-parent="#LinkCheckAboutAccordion">
<div class="accordion-body">
<p>
Depending on the type of messages you are testing, opening all links on all messages
may have undesired consequences:
</p>
<ul>
<li>If the message contains tracking links this may reveal your identity.</li>
<li>
If the message contains unsubscribe links, Link check could unintentionally
unsubscribe you.
</li>
<li>
To speed up the checking process, Link check will attempt 5 URLs at a time. This
could lead to temporary heady load on the remote server.
</li>
</ul>
<p>
Unless you know what messages you receive, it is advised to only run the Link check
manually.
</p>
</div>
</div>
</div>
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn btn-secondary" data-bs-dismiss="modal">Close</button>
</div>
</div>
</div>
</div>
</template>

86
server/ui/api/v1/swagger.json
View File

@ -124,7 +124,7 @@
},
"/api/v1/message/{ID}/html-check": {
"get": {
"description": "Returns the summary of HTML check.\n\nNOTE: This feature is currently in beta and is documented for reference only.\nPlease do not integrate with it (yet) as there may be changes.",
"description": "Returns the summary of the message HTML checker.\n\nNOTE: This feature is currently in beta and is documented for reference only.\nPlease do not integrate with it (yet) as there may be changes.",
"produces": [
"application/json"
],
@ -159,6 +159,50 @@
}
}
},
"/api/v1/message/{ID}/link-check": {
"get": {
"description": "Returns the summary of the message Link checker.\n\nNOTE: This feature is currently in beta and is documented for reference only.\nPlease do not integrate with it (yet) as there may be changes.",
"produces": [
"application/json"
],
"schemes": [
"http",
"https"
],
"tags": [
"Other"
],
"summary": "Link check (beta)",
"operationId": "LinkCheckResponse",
"parameters": [
{
"type": "string",
"description": "Database ID",
"name": "ID",
"in": "path",
"required": true
},
{
"type": "boolean",
"default": false,
"description": "Follow redirects",
"name": "follow",
"in": "query"
}
],
"responses": {
"200": {
"description": "LinkCheckResponse",
"schema": {
"$ref": "#/definitions/LinkCheckResponse"
}
},
"default": {
"$ref": "#/responses/ErrorResponse"
}
}
}
},
"/api/v1/message/{ID}/part/{PartID}": {
"get": {
"description": "This will return the attachment part using the appropriate Content-Type.",
@ -828,6 +872,46 @@
"x-go-name": "Warning",
"x-go-package": "github.com/axllent/mailpit/utils/htmlcheck"
},
"Link": {
"description": "Link struct",
"type": "object",
"properties": {
"Status": {
"description": "HTTP status definition",
"type": "string"
},
"StatusCode": {
"description": "HTTP status code",
"type": "integer",
"format": "int64"
},
"URL": {
"description": "Link URL",
"type": "string"
}
},
"x-go-package": "github.com/axllent/mailpit/utils/linkcheck"
},
"LinkCheckResponse": {
"description": "Response represents the Link check response",
"type": "object",
"properties": {
"Errors": {
"description": "Total number of errors",
"type": "integer",
"format": "int64"
},
"Links": {
"description": "Tested links",
"type": "array",
"items": {
"$ref": "#/definitions/Link"
}
}
},
"x-go-name": "Response",
"x-go-package": "github.com/axllent/mailpit/utils/linkcheck"
},
"Message": {
"description": "Message data excluding physical attachments",
"type": "object",

3
utils/htmlcheck/css.go
View File

@ -11,6 +11,7 @@ import (
"github.com/PuerkitoBio/goquery"
"github.com/axllent/mailpit/config"
"github.com/axllent/mailpit/utils/logger"
"github.com/axllent/mailpit/utils/tools"
"github.com/vanng822/go-premailer/premailer"
"golang.org/x/net/html"
"golang.org/x/net/html/atom"
@ -59,7 +60,7 @@ func runCSSTests(html string) ([]Warning, int, error) {
// get a list of all generated styles from all nodes
allNodeStyles := []string{}
for _, n := range doc.Find("*[style]").Nodes {
style, err := getHTMLAttributeVal(n, "style")
style, err := tools.GetHTMLAttributeVal(n, "style")
if err == nil {
allNodeStyles = append(allNodeStyles, style)
}

14
utils/htmlcheck/html.go
View File

@ -5,7 +5,7 @@ import (
"strings"
"github.com/PuerkitoBio/goquery"
"golang.org/x/net/html"
"github.com/axllent/mailpit/utils/tools"
)
// HTML tests
@ -72,7 +72,7 @@ func runHTMLTests(html string) ([]Warning, int, error) {
totalTests = totalTests + len(imageRegexpTests)
for _, image := range images {
src, err := getHTMLAttributeVal(image, "src")
src, err := tools.GetHTMLAttributeVal(image, "src")
if err != nil {
continue
}
@ -100,13 +100,3 @@ func runHTMLTests(html string) ([]Warning, int, error) {
return results, totalTests, nil
}
func getHTMLAttributeVal(e *html.Node, key string) (string, error) {
for _, a := range e.Attr {
if a.Key == key {
return a.Val, nil
}
}
return "", nil
}

92
utils/linkcheck/main.go Normal file
View File

@ -0,0 +1,92 @@
// Package linkcheck handles message links checking
package linkcheck
import (
"regexp"
"strings"
"github.com/PuerkitoBio/goquery"
"github.com/axllent/mailpit/storage"
"github.com/axllent/mailpit/utils/tools"
)
var linkRe = regexp.MustCompile(`(?m)\b(http|ftp|https):\/\/([\w_-]+(?:(?:\.[\w_-]+)+))([\w.,@?^=%&:'!\/~+#-]*[\w@?^=%&\/~+#-])`)
// RunTests will run all tests on an HTML string
func RunTests(msg *storage.Message, followRedirects bool) (Response, error) {
s := Response{}
allLinks := extractHTMLLinks(msg)
allLinks = strUnique(append(allLinks, extractTextLinks(msg)...))
s.Links = getHTTPStatuses(allLinks, followRedirects)
for _, l := range s.Links {
if l.StatusCode >= 400 || l.StatusCode == 0 {
s.Errors++
}
}
return s, nil
}
func extractTextLinks(msg *storage.Message) []string {
links := []string{}
for _, match := range linkRe.FindAllString(msg.Text, -1) {
links = append(links, match)
}
return links
}
func extractHTMLLinks(msg *storage.Message) []string {
links := []string{}
reader := strings.NewReader(msg.HTML)
// Load the HTML document
doc, err := goquery.NewDocumentFromReader(reader)
if err != nil {
return links
}
aLinks := doc.Find("a[href]").Nodes
for _, link := range aLinks {
l, err := tools.GetHTMLAttributeVal(link, "href")
if err == nil && linkRe.MatchString(l) {
links = append(links, l)
}
}
cssLinks := doc.Find("link[rel=\"stylesheet\"]").Nodes
for _, link := range cssLinks {
l, err := tools.GetHTMLAttributeVal(link, "href")
if err == nil && linkRe.MatchString(l) {
links = append(links, l)
}
}
imgLinks := doc.Find("img[src]").Nodes
for _, link := range imgLinks {
l, err := tools.GetHTMLAttributeVal(link, "src")
if err == nil && linkRe.MatchString(l) {
links = append(links, l)
}
}
return links
}
// strUnique return a slice of unique strings from a slice
func strUnique(strSlice []string) []string {
keys := make(map[string]bool)
list := []string{}
for _, entry := range strSlice {
if _, value := keys[entry]; !value {
keys[entry] = true
list = append(list, entry)
}
}
return list
}

106
utils/linkcheck/status.go Normal file
View File

@ -0,0 +1,106 @@
package linkcheck
import (
"net/http"
"regexp"
"sync"
"time"
"github.com/axllent/mailpit/config"
"github.com/axllent/mailpit/utils/logger"
)
func getHTTPStatuses(links []string, followRedirects bool) []Link {
// allow 5 threads
threads := make(chan int, 5)
results := make(map[string]Link, len(links))
resultsMutex := sync.RWMutex{}
output := []Link{}
var wg sync.WaitGroup
for _, l := range links {
wg.Add(1)
go func(link string, w *sync.WaitGroup) {
threads <- 1 // will block if MAX threads
defer w.Done()
code, err := doHead(link, followRedirects)
l := Link{}
l.URL = link
if err != nil {
l.StatusCode = 0
l.Status = httpErrorSummary(err)
} else {
l.StatusCode = code
l.Status = http.StatusText(code)
}
resultsMutex.Lock()
results[link] = l
resultsMutex.Unlock()
<-threads // remove from threads
}(l, &wg)
}
wg.Wait()
for _, l := range results {
output = append(output, l)
}
return output
}
// Do a HEAD request to return HTTP status code
func doHead(link string, followRedirects bool) (int, error) {
timeout := time.Duration(10 * time.Second)
client := http.Client{
Timeout: timeout,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
if followRedirects {
return nil
}
return http.ErrUseLastResponse
},
}
req, err := http.NewRequest("HEAD", link, nil)
if err != nil {
logger.Log().Error(err)
return 0, err
}
req.Header.Set("User-Agent", "Mailpit/"+config.Version)
res, err := client.Do(req)
if err != nil {
if res != nil {
return res.StatusCode, err
}
return 0, err
}
return res.StatusCode, nil
}
// HTTP errors include a lot more info that just the actual error, so this
// tries to take the final part of it, eg: `no such host`
func httpErrorSummary(err error) string {
var re = regexp.MustCompile(`.*: (.*)$`)
e := err.Error()
if !re.MatchString(e) {
return e
}
parts := re.FindAllStringSubmatch(e, -1)
return parts[0][len(parts[0])-1]
}

21
utils/linkcheck/structs.go Normal file
View File

@ -0,0 +1,21 @@
package linkcheck
// Response represents the Link check response
//
// swagger:model LinkCheckResponse
type Response struct {
// Total number of errors
Errors int `json:"Errors"`
// Tested links
Links []Link `json:"Links"`
}
// Link struct
type Link struct {
// Link URL
URL string `json:"URL"`
// HTTP status code
StatusCode int `json:"StatusCode"`
// HTTP status definition
Status string `json:"Status"`
}

19
utils/tools/html.go Normal file
View File

@ -0,0 +1,19 @@
package tools
import (
"fmt"
"golang.org/x/net/html"
)
// GetHTMLAttributeVal returns the value of an HTML Attribute, else an error.
// Returns a blank value if the attribute is set but empty.
func GetHTMLAttributeVal(e *html.Node, key string) (string, error) {
for _, a := range e.Attr {
if a.Key == key {
return a.Val, nil
}
}
return "", fmt.Errorf("%s not found", key)
}