go/echo
go/echo
1
0
Fork 0
mirror of https://github.com/labstack/echo.git synced 2024-12-24 20:14:31 +02:00
Code Issues Releases Activity

Bugfix/1834 Fix X-Real-IP bug (#2007)

Browse Source 
* Fix incorrect return ip value for RealIpHeader
* Improve test file to compare correct real IPs to each other and have better comments
* Refactor ip extractor tests to be more readable (longer but readable)

Co-authored-by: toimtoimtoim <desinformatsioon@gmail.com>
This commit is contained in:
Yusuf Eyisan 2022-03-01 10:56:46 +03:00 committed by GitHub
parent 27b404bbc5
commit 124825ee62
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 724 additions and 227 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
echo.go
View File

@ -214,9 +214,9 @@ const (
HeaderXForwardedSsl = "X-Forwarded-Ssl"
HeaderXUrlScheme = "X-Url-Scheme"
HeaderXHTTPMethodOverride = "X-HTTP-Method-Override"
HeaderXRealIP = "X-Real-IP"
HeaderXRequestID = "X-Request-ID"
HeaderXCorrelationID = "X-Correlation-ID"
HeaderXRealIP = "X-Real-Ip"
HeaderXRequestID = "X-Request-Id"
HeaderXCorrelationID = "X-Correlation-Id"
HeaderXRequestedWith = "X-Requested-With"
HeaderServer = "Server"
HeaderOrigin = "Origin"

142
ip.go
View File

@ -6,6 +6,130 @@ import (
"strings"
)
/**
By: https://github.com/tmshn (See: https://github.com/labstack/echo/pull/1478 , https://github.com/labstack/echox/pull/134 )
Source: https://echo.labstack.com/guide/ip-address/
IP address plays fundamental role in HTTP; it's used for access control, auditing, geo-based access analysis and more.
Echo provides handy method [`Context#RealIP()`](https://godoc.org/github.com/labstack/echo#Context) for that.
However, it is not trivial to retrieve the _real_ IP address from requests especially when you put L7 proxies before the application.
In such situation, _real_ IP needs to be relayed on HTTP layer from proxies to your app, but you must not trust HTTP headers unconditionally.
Otherwise, you might give someone a chance of deceiving you. **A security risk!**
To retrieve IP address reliably/securely, you must let your application be aware of the entire architecture of your infrastructure.
In Echo, this can be done by configuring `Echo#IPExtractor` appropriately.
This guides show you why and how.
> Note: if you dont' set `Echo#IPExtractor` explicitly, Echo fallback to legacy behavior, which is not a good choice.
Let's start from two questions to know the right direction:
1. Do you put any HTTP (L7) proxy in front of the application?
- It includes both cloud solutions (such as AWS ALB or GCP HTTP LB) and OSS ones (such as Nginx, Envoy or Istio ingress gateway).
2. If yes, what HTTP header do your proxies use to pass client IP to the application?
## Case 1. With no proxy
If you put no proxy (e.g.: directory facing to the internet), all you need to (and have to) see is IP address from network layer.
Any HTTP header is untrustable because the clients have full control what headers to be set.
In this case, use `echo.ExtractIPDirect()`.
```go
e.IPExtractor = echo.ExtractIPDirect()
```
## Case 2. With proxies using `X-Forwarded-For` header
[`X-Forwared-For` (XFF)](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For) is the popular header
to relay clients' IP addresses.
At each hop on the proxies, they append the request IP address at the end of the header.
Following example diagram illustrates this behavior.
```text
┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐
│ "Origin" │───────────>│ Proxy 1 │───────────>│ Proxy 2 │───────────>│ Your app │
│ (IP: a) │ │ (IP: b) │ │ (IP: c) │ │ │
└──────────┘ └──────────┘ └──────────┘ └──────────┘
Case 1.
XFF: "" "a" "a, b"
~~~~~~
Case 2.
XFF: "x" "x, a" "x, a, b"
~~~~~~~~~
↑ What your app will see
```
In this case, use **first _untrustable_ IP reading from right**. Never use first one reading from left, as it is
configurable by client. Here "trustable" means "you are sure the IP address belongs to your infrastructre".
In above example, if `b` and `c` are trustable, the IP address of the client is `a` for both cases, never be `x`.
In Echo, use `ExtractIPFromXFFHeader(...TrustOption)`.
```go
e.IPExtractor = echo.ExtractIPFromXFFHeader()
```
By default, it trusts internal IP addresses (loopback, link-local unicast, private-use and unique local address
from [RFC6890](https://tools.ietf.org/html/rfc6890), [RFC4291](https://tools.ietf.org/html/rfc4291) and
[RFC4193](https://tools.ietf.org/html/rfc4193)).
To control this behavior, use [`TrustOption`](https://godoc.org/github.com/labstack/echo#TrustOption)s.
E.g.:
```go
e.IPExtractor = echo.ExtractIPFromXFFHeader(
TrustLinkLocal(false),
TrustIPRanges(lbIPRange),
)
```
- Ref: https://godoc.org/github.com/labstack/echo#TrustOption
## Case 3. With proxies using `X-Real-IP` header
`X-Real-IP` is another HTTP header to relay clients' IP addresses, but it carries only one address unlike XFF.
If your proxies set this header, use `ExtractIPFromRealIPHeader(...TrustOption)`.
```go
e.IPExtractor = echo.ExtractIPFromRealIPHeader()
```
Again, it trusts internal IP addresses by default (loopback, link-local unicast, private-use and unique local address
from [RFC6890](https://tools.ietf.org/html/rfc6890), [RFC4291](https://tools.ietf.org/html/rfc4291) and
[RFC4193](https://tools.ietf.org/html/rfc4193)).
To control this behavior, use [`TrustOption`](https://godoc.org/github.com/labstack/echo#TrustOption)s.
- Ref: https://godoc.org/github.com/labstack/echo#TrustOption
> **Never forget** to configure the outermost proxy (i.e.; at the edge of your infrastructure) **not to pass through incoming headers**.
> Otherwise there is a chance of fraud, as it is what clients can control.
## About default behavior
In default behavior, Echo sees all of first XFF header, X-Real-IP header and IP from network layer.
As you might already notice, after reading this article, this is not good.
Sole reason this is default is just backward compatibility.
## Private IP ranges
See: https://en.wikipedia.org/wiki/Private_network
Private IPv4 address ranges (RFC 1918):
* 10.0.0.0 – 10.255.255.255 (24-bit block)
* 172.16.0.0 – 172.31.255.255 (20-bit block)
* 192.168.0.0 – 192.168.255.255 (16-bit block)
Private IPv6 address ranges:
* fc00::/7 address block = RFC 4193 Unique Local Addresses (ULA)
*/
type ipChecker struct {
trustLoopback bool
trustLinkLocal bool
@ -52,6 +176,7 @@ func newIPChecker(configs []TrustOption) *ipChecker {
return checker
}
// Go1.16+ added `ip.IsPrivate()` but until that use this implementation
func isPrivateIPRange(ip net.IP) bool {
if ip4 := ip.To4(); ip4 != nil {
return ip4[0] == 10 ||
@ -87,10 +212,12 @@ type IPExtractor func(*http.Request) string
// ExtractIPDirect extracts IP address using actual IP address.
// Use this if your server faces to internet directory (i.e.: uses no proxy).
func ExtractIPDirect() IPExtractor {
return func(req *http.Request) string {
ra, _, _ := net.SplitHostPort(req.RemoteAddr)
return ra
}
return extractIP
}
func extractIP(req *http.Request) string {
ra, _, _ := net.SplitHostPort(req.RemoteAddr)
return ra
}
// ExtractIPFromRealIPHeader extracts IP address using x-real-ip header.
@ -98,14 +225,13 @@ func ExtractIPDirect() IPExtractor {
func ExtractIPFromRealIPHeader(options ...TrustOption) IPExtractor {
checker := newIPChecker(options)
return func(req *http.Request) string {
directIP := ExtractIPDirect()(req)
realIP := req.Header.Get(HeaderXRealIP)
if realIP != "" {
if ip := net.ParseIP(directIP); ip != nil && checker.trust(ip) {
if ip := net.ParseIP(realIP); ip != nil && checker.trust(ip) {
return realIP
}
}
return directIP
return extractIP(req)
}
}
@ -115,7 +241,7 @@ func ExtractIPFromRealIPHeader(options ...TrustOption) IPExtractor {
func ExtractIPFromXFFHeader(options ...TrustOption) IPExtractor {
checker := newIPChecker(options)
return func(req *http.Request) string {
directIP := ExtractIPDirect()(req)
directIP := extractIP(req)
xffs := req.Header[HeaderXForwardedFor]
if len(xffs) == 0 {
return directIP

803
ip_test.go
View File

@ -1,235 +1,606 @@
package echo
import (
"github.com/stretchr/testify/assert"
"net"
"net/http"
"strings"
"testing"
testify "github.com/stretchr/testify/assert"
)
const (
// For RemoteAddr
ipForRemoteAddrLoopback = "127.0.0.1" // From 127.0.0.0/8
sampleRemoteAddrLoopback = ipForRemoteAddrLoopback + ":8080"
ipForRemoteAddrExternal = "203.0.113.1"
sampleRemoteAddrExternal = ipForRemoteAddrExternal + ":8080"
// For x-real-ip
ipForRealIP = "203.0.113.10"
// For XFF
ipForXFF1LinkLocal = "169.254.0.101" // From 169.254.0.0/16
ipForXFF2Private = "192.168.0.102" // From 192.168.0.0/16
ipForXFF3External = "2001:db8::103"
ipForXFF4Private = "fc00::104" // From fc00::/7
ipForXFF5External = "198.51.100.105"
ipForXFF6External = "192.0.2.106"
ipForXFFBroken = "this.is.broken.lol"
// keys for test cases
ipTestReqKeyNoHeader = "no header"
ipTestReqKeyRealIPExternal = "x-real-ip; remote addr external"
ipTestReqKeyRealIPInternal = "x-real-ip; remote addr internal"
ipTestReqKeyRealIPAndXFFExternal = "x-real-ip and xff; remote addr external"
ipTestReqKeyRealIPAndXFFInternal = "x-real-ip and xff; remote addr internal"
ipTestReqKeyXFFExternal = "xff; remote addr external"
ipTestReqKeyXFFInternal = "xff; remote addr internal"
ipTestReqKeyBrokenXFF = "broken xff"
)
var (
sampleXFF = strings.Join([]string{
ipForXFF6External, ipForXFF5External, ipForXFF4Private, ipForXFF3External, ipForXFF2Private, ipForXFF1LinkLocal,
}, ", ")
requests = map[string]*http.Request{
ipTestReqKeyNoHeader: &http.Request{
RemoteAddr: sampleRemoteAddrExternal,
},
ipTestReqKeyRealIPExternal: &http.Request{
Header: http.Header{
"X-Real-Ip": []string{ipForRealIP},
},
RemoteAddr: sampleRemoteAddrExternal,
},
ipTestReqKeyRealIPInternal: &http.Request{
Header: http.Header{
"X-Real-Ip": []string{ipForRealIP},
},
RemoteAddr: sampleRemoteAddrLoopback,
},
ipTestReqKeyRealIPAndXFFExternal: &http.Request{
Header: http.Header{
"X-Real-Ip": []string{ipForRealIP},
HeaderXForwardedFor: []string{sampleXFF},
},
RemoteAddr: sampleRemoteAddrExternal,
},
ipTestReqKeyRealIPAndXFFInternal: &http.Request{
Header: http.Header{
"X-Real-Ip": []string{ipForRealIP},
HeaderXForwardedFor: []string{sampleXFF},
},
RemoteAddr: sampleRemoteAddrLoopback,
},
ipTestReqKeyXFFExternal: &http.Request{
Header: http.Header{
HeaderXForwardedFor: []string{sampleXFF},
},
RemoteAddr: sampleRemoteAddrExternal,
},
ipTestReqKeyXFFInternal: &http.Request{
Header: http.Header{
HeaderXForwardedFor: []string{sampleXFF},
},
RemoteAddr: sampleRemoteAddrLoopback,
},
ipTestReqKeyBrokenXFF: &http.Request{
Header: http.Header{
HeaderXForwardedFor: []string{ipForXFFBroken + ", " + ipForXFF1LinkLocal},
},
RemoteAddr: sampleRemoteAddrLoopback,
},
func mustParseCIDR(s string) *net.IPNet {
_, IPNet, err := net.ParseCIDR(s)
if err != nil {
panic(err)
}
)
return IPNet
}
func TestExtractIP(t *testing.T) {
_, ipv4AllRange, _ := net.ParseCIDR("0.0.0.0/0")
_, ipv6AllRange, _ := net.ParseCIDR("::/0")
_, ipForXFF3ExternalRange, _ := net.ParseCIDR(ipForXFF3External + "/48")
_, ipForRemoteAddrExternalRange, _ := net.ParseCIDR(ipForRemoteAddrExternal + "/24")
tests := map[string]*struct {
extractor IPExtractor
expectedIPs map[string]string
func TestIPChecker_TrustOption(t *testing.T) {
var testCases = []struct {
name string
givenOptions []TrustOption
whenIP string
expect bool
}{
"ExtractIPDirect": {
ExtractIPDirect(),
map[string]string{
ipTestReqKeyNoHeader: ipForRemoteAddrExternal,
ipTestReqKeyRealIPExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPInternal: ipForRemoteAddrLoopback,
ipTestReqKeyRealIPAndXFFExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPAndXFFInternal: ipForRemoteAddrLoopback,
ipTestReqKeyXFFExternal: ipForRemoteAddrExternal,
ipTestReqKeyXFFInternal: ipForRemoteAddrLoopback,
ipTestReqKeyBrokenXFF: ipForRemoteAddrLoopback,
{
name: "ip is within trust range, trusts additional private IPV6 network",
givenOptions: []TrustOption{
TrustLoopback(false),
TrustLinkLocal(false),
TrustPrivateNet(false),
// this is private IPv6 ip
// CIDR Notation: 2001:0db8:0000:0000:0000:0000:0000:0000/48
// Address: 2001:0db8:0000:0000:0000:0000:0000:0103
// Range start: 2001:0db8:0000:0000:0000:0000:0000:0000
// Range end: 2001:0db8:0000:ffff:ffff:ffff:ffff:ffff
TrustIPRange(mustParseCIDR("2001:db8::103/48")),
},
whenIP: "2001:0db8:0000:0000:0000:0000:0000:0103",
expect: true,
},
"ExtractIPFromRealIPHeader(default)": {
ExtractIPFromRealIPHeader(),
map[string]string{
ipTestReqKeyNoHeader: ipForRemoteAddrExternal,
ipTestReqKeyRealIPExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPInternal: ipForRealIP,
ipTestReqKeyRealIPAndXFFExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPAndXFFInternal: ipForRealIP,
ipTestReqKeyXFFExternal: ipForRemoteAddrExternal,
ipTestReqKeyXFFInternal: ipForRemoteAddrLoopback,
ipTestReqKeyBrokenXFF: ipForRemoteAddrLoopback,
},
},
"ExtractIPFromRealIPHeader(trust only direct-facing proxy)": {
ExtractIPFromRealIPHeader(TrustLoopback(false), TrustLinkLocal(false), TrustPrivateNet(false), TrustIPRange(ipForRemoteAddrExternalRange)),
map[string]string{
ipTestReqKeyNoHeader: ipForRemoteAddrExternal,
ipTestReqKeyRealIPExternal: ipForRealIP,
ipTestReqKeyRealIPInternal: ipForRemoteAddrLoopback,
ipTestReqKeyRealIPAndXFFExternal: ipForRealIP,
ipTestReqKeyRealIPAndXFFInternal: ipForRemoteAddrLoopback,
ipTestReqKeyXFFExternal: ipForRemoteAddrExternal,
ipTestReqKeyXFFInternal: ipForRemoteAddrLoopback,
ipTestReqKeyBrokenXFF: ipForRemoteAddrLoopback,
},
},
"ExtractIPFromRealIPHeader(trust direct-facing proxy)": {
ExtractIPFromRealIPHeader(TrustIPRange(ipForRemoteAddrExternalRange)),
map[string]string{
ipTestReqKeyNoHeader: ipForRemoteAddrExternal,
ipTestReqKeyRealIPExternal: ipForRealIP,
ipTestReqKeyRealIPInternal: ipForRealIP,
ipTestReqKeyRealIPAndXFFExternal: ipForRealIP,
ipTestReqKeyRealIPAndXFFInternal: ipForRealIP,
ipTestReqKeyXFFExternal: ipForRemoteAddrExternal,
ipTestReqKeyXFFInternal: ipForRemoteAddrLoopback,
ipTestReqKeyBrokenXFF: ipForRemoteAddrLoopback,
},
},
"ExtractIPFromXFFHeader(default)": {
ExtractIPFromXFFHeader(),
map[string]string{
ipTestReqKeyNoHeader: ipForRemoteAddrExternal,
ipTestReqKeyRealIPExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPInternal: ipForRemoteAddrLoopback,
ipTestReqKeyRealIPAndXFFExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPAndXFFInternal: ipForXFF3External,
ipTestReqKeyXFFExternal: ipForRemoteAddrExternal,
ipTestReqKeyXFFInternal: ipForXFF3External,
ipTestReqKeyBrokenXFF: ipForRemoteAddrLoopback,
},
},
"ExtractIPFromXFFHeader(trust only direct-facing proxy)": {
ExtractIPFromXFFHeader(TrustLoopback(false), TrustLinkLocal(false), TrustPrivateNet(false), TrustIPRange(ipForRemoteAddrExternalRange)),
map[string]string{
ipTestReqKeyNoHeader: ipForRemoteAddrExternal,
ipTestReqKeyRealIPExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPInternal: ipForRemoteAddrLoopback,
ipTestReqKeyRealIPAndXFFExternal: ipForXFF1LinkLocal,
ipTestReqKeyRealIPAndXFFInternal: ipForRemoteAddrLoopback,
ipTestReqKeyXFFExternal: ipForXFF1LinkLocal,
ipTestReqKeyXFFInternal: ipForRemoteAddrLoopback,
ipTestReqKeyBrokenXFF: ipForRemoteAddrLoopback,
},
},
"ExtractIPFromXFFHeader(trust direct-facing proxy)": {
ExtractIPFromXFFHeader(TrustIPRange(ipForRemoteAddrExternalRange)),
map[string]string{
ipTestReqKeyNoHeader: ipForRemoteAddrExternal,
ipTestReqKeyRealIPExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPInternal: ipForRemoteAddrLoopback,
ipTestReqKeyRealIPAndXFFExternal: ipForXFF3External,
ipTestReqKeyRealIPAndXFFInternal: ipForXFF3External,
ipTestReqKeyXFFExternal: ipForXFF3External,
ipTestReqKeyXFFInternal: ipForXFF3External,
ipTestReqKeyBrokenXFF: ipForRemoteAddrLoopback,
},
},
"ExtractIPFromXFFHeader(trust everything)": {
// This is similar to legacy behavior, but ignores x-real-ip header.
ExtractIPFromXFFHeader(TrustIPRange(ipv4AllRange), TrustIPRange(ipv6AllRange)),
map[string]string{
ipTestReqKeyNoHeader: ipForRemoteAddrExternal,
ipTestReqKeyRealIPExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPInternal: ipForRemoteAddrLoopback,
ipTestReqKeyRealIPAndXFFExternal: ipForXFF6External,
ipTestReqKeyRealIPAndXFFInternal: ipForXFF6External,
ipTestReqKeyXFFExternal: ipForXFF6External,
ipTestReqKeyXFFInternal: ipForXFF6External,
ipTestReqKeyBrokenXFF: ipForRemoteAddrLoopback,
},
},
"ExtractIPFromXFFHeader(trust ipForXFF3External)": {
// This trusts private network also after "additional" trust ranges unlike `TrustNProxies(1)` doesn't
ExtractIPFromXFFHeader(TrustIPRange(ipForXFF3ExternalRange)),
map[string]string{
ipTestReqKeyNoHeader: ipForRemoteAddrExternal,
ipTestReqKeyRealIPExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPInternal: ipForRemoteAddrLoopback,
ipTestReqKeyRealIPAndXFFExternal: ipForRemoteAddrExternal,
ipTestReqKeyRealIPAndXFFInternal: ipForXFF5External,
ipTestReqKeyXFFExternal: ipForRemoteAddrExternal,
ipTestReqKeyXFFInternal: ipForXFF5External,
ipTestReqKeyBrokenXFF: ipForRemoteAddrLoopback,
{
name: "ip is within trust range, trusts additional private IPV6 network",
givenOptions: []TrustOption{
TrustIPRange(mustParseCIDR("2001:db8::103/48")),
},
whenIP: "2001:0db8:0000:0000:0000:0000:0000:0103",
expect: true,
},
}
for name, test := range tests {
t.Run(name, func(t *testing.T) {
assert := testify.New(t)
for key, req := range requests {
actual := test.extractor(req)
expected := test.expectedIPs[key]
assert.Equal(expected, actual, "Request: %s", key)
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
checker := newIPChecker(tc.givenOptions)
result := checker.trust(net.ParseIP(tc.whenIP))
assert.Equal(t, tc.expect, result)
})
}
}
func TestTrustIPRange(t *testing.T) {
var testCases = []struct {
name string
givenRange string
whenIP string
expect bool
}{
{
name: "ip is within trust range, IPV6 network range",
// CIDR Notation: 2001:0db8:0000:0000:0000:0000:0000:0000/48
// Address: 2001:0db8:0000:0000:0000:0000:0000:0103
// Range start: 2001:0db8:0000:0000:0000:0000:0000:0000
// Range end: 2001:0db8:0000:ffff:ffff:ffff:ffff:ffff
givenRange: "2001:db8::103/48",
whenIP: "2001:0db8:0000:0000:0000:0000:0000:0103",
expect: true,
},
{
name: "ip is outside (upper bounds) of trust range, IPV6 network range",
givenRange: "2001:db8::103/48",
whenIP: "2001:0db8:0001:0000:0000:0000:0000:0000",
expect: false,
},
{
name: "ip is outside (lower bounds) of trust range, IPV6 network range",
givenRange: "2001:db8::103/48",
whenIP: "2001:0db7:ffff:ffff:ffff:ffff:ffff:ffff",
expect: false,
},
{
name: "ip is within trust range, IPV4 network range",
// CIDR Notation: 8.8.8.8/24
// Address: 8.8.8.8
// Range start: 8.8.8.0
// Range end: 8.8.8.255
givenRange: "8.8.8.0/24",
whenIP: "8.8.8.8",
expect: true,
},
{
name: "ip is within trust range, IPV4 network range",
// CIDR Notation: 8.8.8.8/24
// Address: 8.8.8.8
// Range start: 8.8.8.0
// Range end: 8.8.8.255
givenRange: "8.8.8.0/24",
whenIP: "8.8.8.8",
expect: true,
},
{
name: "ip is outside (upper bounds) of trust range, IPV4 network range",
givenRange: "8.8.8.0/24",
whenIP: "8.8.9.0",
expect: false,
},
{
name: "ip is outside (lower bounds) of trust range, IPV4 network range",
givenRange: "8.8.8.0/24",
whenIP: "8.8.7.255",
expect: false,
},
{
name: "public ip, trust everything in IPV4 network range",
givenRange: "0.0.0.0/0",
whenIP: "8.8.8.8",
expect: true,
},
{
name: "internal ip, trust everything in IPV4 network range",
givenRange: "0.0.0.0/0",
whenIP: "127.0.10.1",
expect: true,
},
{
name: "public ip, trust everything in IPV6 network range",
givenRange: "::/0",
whenIP: "2a00:1450:4026:805::200e",
expect: true,
},
{
name: "internal ip, trust everything in IPV6 network range",
givenRange: "::/0",
whenIP: "0:0:0:0:0:0:0:1",
expect: true,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
cidr := mustParseCIDR(tc.givenRange)
checker := newIPChecker([]TrustOption{
TrustLoopback(false), // disable to avoid interference
TrustLinkLocal(false), // disable to avoid interference
TrustPrivateNet(false), // disable to avoid interference
TrustIPRange(cidr),
})
result := checker.trust(net.ParseIP(tc.whenIP))
assert.Equal(t, tc.expect, result)
})
}
}
func TestTrustPrivateNet(t *testing.T) {
var testCases = []struct {
name string
whenIP string
expect bool
}{
{
name: "do not trust public IPv4 address",
whenIP: "8.8.8.8",
expect: false,
},
{
name: "do not trust public IPv6 address",
whenIP: "2a00:1450:4026:805::200e",
expect: false,
},
{ // Class A: 10.0.0.0 — 10.255.255.255
name: "do not trust IPv4 just outside of class A (lower bounds)",
whenIP: "9.255.255.255",
expect: false,
},
{
name: "do not trust IPv4 just outside of class A (upper bounds)",
whenIP: "11.0.0.0",
expect: false,
},
{
name: "trust IPv4 of class A (lower bounds)",
whenIP: "10.0.0.0",
expect: true,
},
{
name: "trust IPv4 of class A (upper bounds)",
whenIP: "10.255.255.255",
expect: true,
},
{ // Class B: 172.16.0.0 — 172.31.255.255
name: "do not trust IPv4 just outside of class B (lower bounds)",
whenIP: "172.15.255.255",
expect: false,
},
{
name: "do not trust IPv4 just outside of class B (upper bounds)",
whenIP: "172.32.0.0",
expect: false,
},
{
name: "trust IPv4 of class B (lower bounds)",
whenIP: "172.16.0.0",
expect: true,
},
{
name: "trust IPv4 of class B (upper bounds)",
whenIP: "172.31.255.255",
expect: true,
},
{ // Class C: 192.168.0.0 — 192.168.255.255
name: "do not trust IPv4 just outside of class C (lower bounds)",
whenIP: "192.167.255.255",
expect: false,
},
{
name: "do not trust IPv4 just outside of class C (upper bounds)",
whenIP: "192.169.0.0",
expect: false,
},
{
name: "trust IPv4 of class C (lower bounds)",
whenIP: "192.168.0.0",
expect: true,
},
{
name: "trust IPv4 of class C (upper bounds)",
whenIP: "192.168.255.255",
expect: true,
},
{ // fc00::/7 address block = RFC 4193 Unique Local Addresses (ULA)
// splits the address block in two equally sized halves, fc00::/8 and fd00::/8.
// https://en.wikipedia.org/wiki/Unique_local_address
name: "trust IPv6 private address",
whenIP: "fdfc:3514:2cb3:4bd5::",
expect: true,
},
{
name: "do not trust IPv6 just out of /fd (upper bounds)",
whenIP: "/fe00:0000:0000:0000:0000",
expect: false,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
checker := newIPChecker([]TrustOption{
TrustLoopback(false), // disable to avoid interference
TrustLinkLocal(false), // disable to avoid interference
TrustPrivateNet(true),
})
result := checker.trust(net.ParseIP(tc.whenIP))
assert.Equal(t, tc.expect, result)
})
}
}
func TestTrustLinkLocal(t *testing.T) {
var testCases = []struct {
name string
whenIP string
expect bool
}{
{
name: "trust link local IPv4 address (lower bounds)",
whenIP: "169.254.0.0",
expect: true,
},
{
name: "trust link local IPv4 address (upper bounds)",
whenIP: "169.254.255.255",
expect: true,
},
{
name: "do not trust link local IPv4 address (outside of lower bounds)",
whenIP: "169.253.255.255",
expect: false,
},
{
name: "do not trust link local IPv4 address (outside of upper bounds)",
whenIP: "169.255.0.0",
expect: false,
},
{
name: "trust link local IPv6 address ",
whenIP: "fe80::1",
expect: true,
},
{
name: "do not trust link local IPv6 address ",
whenIP: "fec0::1",
expect: false,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
checker := newIPChecker([]TrustOption{
TrustLoopback(false), // disable to avoid interference
TrustPrivateNet(false), // disable to avoid interference
TrustLinkLocal(true),
})
result := checker.trust(net.ParseIP(tc.whenIP))
assert.Equal(t, tc.expect, result)
})
}
}
func TestTrustLoopback(t *testing.T) {
var testCases = []struct {
name string
whenIP string
expect bool
}{
{
name: "trust IPv4 as localhost",
whenIP: "127.0.0.1",
expect: true,
},
{
name: "trust IPv6 as localhost",
whenIP: "::1",
expect: true,
},
{
name: "do not trust public ip as localhost",
whenIP: "8.8.8.8",
expect: false,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
checker := newIPChecker([]TrustOption{
TrustLinkLocal(false), // disable to avoid interference
TrustPrivateNet(false), // disable to avoid interference
TrustLoopback(true),
})
result := checker.trust(net.ParseIP(tc.whenIP))
assert.Equal(t, tc.expect, result)
})
}
}
func TestExtractIPDirect(t *testing.T) {
var testCases = []struct {
name string
whenRequest http.Request
expectIP string
}{
{
name: "request has no headers, extracts IP from request remote addr",
whenRequest: http.Request{
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.1",
},
{
name: "request is from external IP has X-Real-Ip header, extractor still extracts IP from request remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXRealIP: []string{"203.0.113.10"},
},
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.1",
},
{
name: "request is from internal IP and has Real-IP header, extractor still extracts internal IP from request remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXRealIP: []string{"203.0.113.10"},
},
RemoteAddr: "127.0.0.1:8080",
},
expectIP: "127.0.0.1",
},
{
name: "request is from external IP and has XFF + Real-IP header, extractor still extracts external IP from request remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXRealIP: []string{"203.0.113.10"},
HeaderXForwardedFor: []string{"192.0.2.106, 198.51.100.105, fc00::104, 2001:db8::103, 192.168.0.102, 169.254.0.101"},
},
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.1",
},
{
name: "request is from internal IP and has XFF + Real-IP header, extractor still extracts internal IP from request remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXRealIP: []string{"127.0.0.1"},
HeaderXForwardedFor: []string{"192.0.2.106, 198.51.100.105, fc00::104, 2001:db8::103, 192.168.0.102, 169.254.0.101"},
},
RemoteAddr: "127.0.0.1:8080",
},
expectIP: "127.0.0.1",
},
{
name: "request is from external IP and has XFF header, extractor still extracts external IP from request remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXForwardedFor: []string{"192.0.2.106, 198.51.100.105, fc00::104, 2001:db8::103, 192.168.0.102, 169.254.0.101"},
},
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.1",
},
{
name: "request is from internal IP and has XFF header, extractor still extracts internal IP from request remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXForwardedFor: []string{"192.0.2.106, 198.51.100.105, fc00::104, 2001:db8::103, 192.168.0.102, 169.254.0.101"},
},
RemoteAddr: "127.0.0.1:8080",
},
expectIP: "127.0.0.1",
},
{
name: "request is from internal IP and has INVALID XFF header, extractor still extracts internal IP from request remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXForwardedFor: []string{"this.is.broken.lol, 169.254.0.101"},
},
RemoteAddr: "127.0.0.1:8080",
},
expectIP: "127.0.0.1",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
extractedIP := ExtractIPDirect()(&tc.whenRequest)
assert.Equal(t, tc.expectIP, extractedIP)
})
}
}
func TestExtractIPFromRealIPHeader(t *testing.T) {
_, ipForRemoteAddrExternalRange, _ := net.ParseCIDR("203.0.113.199/24")
var testCases = []struct {
name string
givenTrustOptions []TrustOption
whenRequest http.Request
expectIP string
}{
{
name: "request has no headers, extracts IP from request remote addr",
whenRequest: http.Request{
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.1",
},
{
name: "request is from external IP has INVALID external X-Real-Ip header, extract IP from remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXRealIP: []string{"xxx.yyy.zzz.ccc"}, // <-- this is invalid
},
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.1",
},
{
name: "request is from external IP has valid + UNTRUSTED external X-Real-Ip header, extract IP from remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXRealIP: []string{"203.0.113.199"}, // <-- this is untrusted
},
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.1",
},
{
name: "request is from external IP has valid + TRUSTED X-Real-Ip header, extract IP from X-Real-Ip header",
givenTrustOptions: []TrustOption{ // case for "trust direct-facing proxy"
TrustIPRange(ipForRemoteAddrExternalRange), // we trust external IP range "203.0.113.199/24"
},
whenRequest: http.Request{
Header: http.Header{
HeaderXRealIP: []string{"203.0.113.199"},
},
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.199",
},
{
name: "request is from external IP has XFF and valid + TRUSTED X-Real-Ip header, extract IP from X-Real-Ip header",
givenTrustOptions: []TrustOption{ // case for "trust direct-facing proxy"
TrustIPRange(ipForRemoteAddrExternalRange), // we trust external IP range "203.0.113.199/24"
},
whenRequest: http.Request{
Header: http.Header{
HeaderXRealIP: []string{"203.0.113.199"},
HeaderXForwardedFor: []string{"203.0.113.198, 203.0.113.197"}, // <-- should not affect anything
},
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.199",
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
extractedIP := ExtractIPFromRealIPHeader(tc.givenTrustOptions...)(&tc.whenRequest)
assert.Equal(t, tc.expectIP, extractedIP)
})
}
}
func TestExtractIPFromXFFHeader(t *testing.T) {
_, ipForRemoteAddrExternalRange, _ := net.ParseCIDR("203.0.113.199/24")
var testCases = []struct {
name string
givenTrustOptions []TrustOption
whenRequest http.Request
expectIP string
}{
{
name: "request has no headers, extracts IP from request remote addr",
whenRequest: http.Request{
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.1",
},
{
name: "request has INVALID external XFF header, extract IP from remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXForwardedFor: []string{"xxx.yyy.zzz.ccc, 127.0.0.2"}, // <-- this is invalid
},
RemoteAddr: "127.0.0.1:8080",
},
expectIP: "127.0.0.1",
},
{
name: "request trusts all IPs in XFF header, extract IP from furthest in XFF chain",
whenRequest: http.Request{
Header: http.Header{
HeaderXForwardedFor: []string{"127.0.0.3, 127.0.0.2, 127.0.0.1"},
},
RemoteAddr: "127.0.0.1:8080",
},
expectIP: "127.0.0.3",
},
{
name: "request is from external IP has valid + UNTRUSTED external XFF header, extract IP from remote addr",
whenRequest: http.Request{
Header: http.Header{
HeaderXForwardedFor: []string{"203.0.113.199"}, // <-- this is untrusted
},
RemoteAddr: "203.0.113.1:8080",
},
expectIP: "203.0.113.1",
},
{
name: "request is from external IP is valid and has some IPs TRUSTED XFF header, extract IP from XFF header",
givenTrustOptions: []TrustOption{
TrustIPRange(ipForRemoteAddrExternalRange), // we trust external IP range "203.0.113.199/24"
},
// from request its seems that request has been proxied through 6 servers.
// 1) 203.0.1.100 (this is external IP set by 203.0.100.100 which we do not trust - could be spoofed)
// 2) 203.0.100.100 (this is outside of our network but set by 203.0.113.199 which we trust to set correct IPs)
// 3) 203.0.113.199 (we trust, for example maybe our proxy from some other office)
// 4) 192.168.1.100 (internal IP, some internal upstream loadbalancer ala SSL offloading with F5 products)
// 5) 127.0.0.1 (is proxy on localhost. maybe we have Nginx in front of our Echo instance doing some routing)
whenRequest: http.Request{
Header: http.Header{
HeaderXForwardedFor: []string{"203.0.1.100, 203.0.100.100, 203.0.113.199, 192.168.1.100"},
},
RemoteAddr: "127.0.0.1:8080", // IP of proxy upstream of our APP
},
expectIP: "203.0.100.100", // this is first trusted IP in XFF chain
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
extractedIP := ExtractIPFromXFFHeader(tc.givenTrustOptions...)(&tc.whenRequest)
assert.Equal(t, tc.expectIP, extractedIP)
})
}
}