Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add Forwarded header parsing and real IP extraction with tests #2744

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

debug-ing
Copy link

For issue #2694

  • Added parser for the "Forwarded" header to extract the "for" field.
  • Implemented real IP extraction from the "Forwarded" headers.
  • Added unit tests to validate header parsing and real IP extraction functionality.

@debug-ing
Copy link
Author

cc : @aldas

@debug-ing
Copy link
Author

@aldas fixed lint error

@debug-ing
Copy link
Author

debug-ing commented Feb 12, 2025

cc: @aldas

1 similar comment
@debug-ing
Copy link
Author

cc: @aldas

@aldas
Copy link
Contributor

aldas commented Feb 15, 2025

Just for history sake. I did some research myself to understand what this header is and how it is implemented elsewhere

  1. RFC 7239 for this Forwarded header
    https://datatracker.ietf.org/doc/html/rfc7239

  2. Mozilla explanation of that header
    https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded

  3. Commit that add this header support to HAproxy
    haproxy/haproxy@b2bb925 HAproxy

    Current implementation proxy_http_parse_7239
    https://github.com/haproxy/haproxy/blob/32691e7c255a944224e0ac17d0f83f217be6c5d6/src/http_ext.c#L963

  4. Proposal to Go standard library proposal: net/http/httputil: support RFC 7239 Forwarded header in ReverseProxy
    proposal: net/http/httputil: support RFC 7239 Forwarded header in ReverseProxy golang/go#20526

  5. old Go library that implements Forwarded parsing
    https://github.com/stanvit/go-forwarded

  6. Discussion for Caddy to add this header support
    Add support for Forwarded header (RFC 7239) caddyserver/caddy#3262

  7. Nginx wiki about Forwarded header
    https://github.com/nginxinc/nginx-wiki/blob/master/source/start/topics/examples/forwarded.rst "Using the Forwarded header"
    https://trac.nginx.org/nginx/ticket/1316

  8. PHP framework: Symphony commit to add support for header
    Added new Forwarded header support for Request::getClientIps symfony/symfony#11379

  9. Previous PR for that header in Echo
    Fix issue #2694 #2702

@@ -40,6 +40,8 @@ type Context interface {
// Scheme returns the HTTP protocol scheme, `http` or `https`.
Scheme() string

SchemeForwarded() *Forwarded
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding new SchemeForwarded method to Context interface is a backwards incompatible change. It has to be some utility function or specific IPExtractor implementation

func (c *context) RealIP() string {
if c.echo != nil && c.echo.IPExtractor != nil {
return c.echo.IPExtractor(c.request)
}
// Check if the "Forwarded" header is present in the request.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After seeing how complex or how less understood Forwarded header is - I think it is better not to add it directly to RealIP method. It is better suited as separate IPExtractor implementation

func (c *context) SchemeForwarded() *Forwarded {
// Parse and get "Forwarded" header.
// See : https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded
if scheme := c.request.Header.Get(HeaderForwarded); scheme != "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only parses single Header. From mozilla article I understand that you can have actually multiple Forwarded Headers in request.

RFC says https://datatracker.ietf.org/doc/html/rfc7239

A proxy server that wants to add a new "Forwarded" header field value
can either append it to the last existing "Forwarded" header field
after a comma separator or add a new field at the end of the header
block
.

Mozilla rephrases this as
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Forwarded

If there are multiple proxy servers between the client and server, they may each specify their own forwarding information. This can be done by adding a new Forwarded header to the end of the header block, or by appending the information to the end of the last Forwarded header in a comma-separated list.

There is Nginx issue https://trac.nginx.org/nginx/ticket/1316 that has example for multiple headers in request


func (c *context) parseForwarded(input string) (Forwarded, error) {
forwarded := Forwarded{}
entries := strings.Split(input, ",")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest - I do not understand when is semicolon used and when is colon used.
Mozilla documentation says that Directives are key=value pairs, separated by a semicolon. and This can be done by adding a new Forwarded header to the end of the header block, or by appending the information to the end of the last Forwarded header in a comma-separated list.

Can semicolon and comma be mixed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally - what about quoted strings that contain comma or semicolon?
RFC says that field value can be value = token / quoted-string where

These 2 are defined as

token = <Defined in [RFC7230], Section 3.2.6>
quoted-string = <Defined in [RFC7230], Section 3.2.6>

I think rfc7230 section-3.2.6 allows unescaped commas in quoted field value

}

key := strings.TrimSpace(parts[0])
value, err := url.QueryUnescape(strings.TrimSpace(parts[1]))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is url.QueryUnescape correct by rfc7230 section-3.2.6?

RFC says that value is defined as:

forwarded-pair = token "=" value
value = token / quoted-string
token = <Defined in [RFC7230], Section 3.2.6>
quoted-string = <Defined in [RFC7230], Section 3.2.6>

@aldas
Copy link
Contributor

aldas commented Feb 15, 2025

This probably be better off as separate library

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants