What sequence of API calls should be made when calling through HTTP Proxy?

[shahamit]

I am using openresty http library (v0.16.1) inside a custom kong plugin. I need to make an api call through a http proxy if it is configured. So my current code looks as below -
The sequence of api calls made when going through proxy are

1. set_proxy_options
2. request_uri

While the sequence of api calls made when directly calling the remote host are

1. connect
2. ssl_handshake
3. request

Below are my questions

1. Are the api call sequences accurate in both the scenarios?
2. Why don't we need to call connect when going through the proxy?
3. Should ssl handshaking be done when going through the proxy? If so, how?

Thanks.

    if conf.proxy_uri then
        kong.log.notice("Setting the proxy options")
        local proxy_opts = {
            http_proxy = conf.proxy_uri,
            https_proxy = conf.proxy_uri
        }
        httpc:set_proxy_options(proxy_opts)
        kong.log.notice("Making http request through proxy")
        res, err = httpc:request_uri(conf.api_path_, auth_request)
        _M.process_response(res, res.body, err)
    else
        kong.log.notice("Connecting directly to the remote host")
        httpc:connect(host, port)
        if scheme == "https" then
            ok, err = httpc:ssl_handshake()
            if not ok then
                kong.log.err(err)
                return kong.response.exit(500, { message = "An unexpected error occurred during ssl handshake."})
            end
        end
        res, err = httpc:request(auth_request)
        _M.process_response(res, res:read_body(), err)
    end

[pintsized]

I think perhaps you are somehow confusing "single-shot vs streaming", with "proxy vs direct".

The request_uri interface is for making single-shot requests - that is, requests which will connect, read everything (including the body), and then close or set_keepalive depending on the configuration. The request interface (otherwise known as "generic") allows you to stream the response body if and when you are ready to do so (it only reads status and headers automatically), but requires you to manually call connect and set_keepalive etc.

You are able to use set_proxy_options equally in both cases, and everything should just work out of the box (including ssl handshake, and so on).

[shahamit]

Ok I understand. So theoretically there should be only one call to set_proxy_options whenever the call needs to be routed through the proxy. Rest of the code should be the same with or without proxy, right?

I tried the below code but it fails when calling the request api. The error is

2023/03/10 10:02:32 [notice] 7759#0: *279063 [kong] access.lua:46 [kong-uam-plugin] Making http request, client: 172.20.0.1, server: kong, request: "GET /api/leads/v1/business-entities HTTP/1.1", host: "localhost:8000"
2023/03/10 10:02:32 [error] 7759#0: *279063 [kong] init.lua:297 [kong-uam-plugin] /usr/local/share/lua/5.1/resty/http.lua:697: bad argument #1 to 'str_sub' (string expected, got nil), client: 172.20.0.1, server: kong, request: "GET /api/leads/v1/business-entities HTTP/1.1", host: "localhost:8000"

I am unable to figure out what is the problem.

The code is as below

    if conf.proxy_uri then
        kong.log.notice("Setting the proxy options")
        local proxy_opts = {
            http_proxy = conf.proxy_uri,
            https_proxy = conf.proxy_uri
        }
        httpc:set_proxy_options(proxy_opts)
        kong.log.notice("Setting the proxy options....done")
    end
    local connect_opts = {
        scheme = scheme,
        host = host,
        port = port
    }
    httpc:connect(connect_opts)
    kong.log.notice("Making http request")
    res, err = httpc:request(auth_request)

[pintsized]

Well, the error message shows that the code fails when trying to treat a variable as a string, when it is nil. And if you inspect the code, that variable is self.host. Which suggests you are setting host to nil in your options. But I can't see this for sure because you have not posted a complete example.

I guess we should detect something as simple as that to give a better error message, instead of crashing. PR's welcome!

p.s, the idea with the new (smarter) connect mechanism is that you should just be able to put all of your config in one table, and call connect(opts). If the proxy options are non-nil, they will be used. So you can avoid the if statement and the call to set_proxy_options, I'm pretty sure. Just make sure your connect_opts table looks sane (debug log and check!), and all should work fine.

p.p.s, there are definitely other Kong plugins that do this. Perhaps they are enterprise only, I don't remember. But you might find working example code if you look around.

[shahamit]

Thanks for helping on the troubleshooting but I verified the variable values - scheme, host and port. None of them are nil. Also the connect call succeeds but here the httpc:request call is failing.

I didn't post the complete code because it has much more other things that would confuse the discussion. I am sharing the recent logs and the code for ease.

2023/03/10 11:20:47 [notice] 13934#0: *504227 [kong] access.lua:39 [kong-uam-plugin] Scheme : https Host : ssgsit.serviceurl.in Port : 443, client: 172.20.0.1, server: kong, request: "GET /api/leads/v1/business-entities HTTP/1.1", host: "localhost:8000"
2023/03/10 11:20:47 [notice] 13934#0: *504227 [kong] access.lua:46 [kong-uam-plugin] Making http request, client: 172.20.0.1, server: kong, request: "GET /api/leads/v1/business-entities HTTP/1.1", host: "localhost:8000"
2023/03/10 11:20:47 [error] 13934#0: *504227 [kong] init.lua:297 [kong-uam-plugin] /usr/local/share/lua/5.1/resty/http.lua:697: bad argument #1 to 'str_sub' (string expected, got nil), client: 172.20.0.1, server: kong, request: "GET /api/leads/v1/business-entities HTTP/1.1", host: "localhost:8000"
172.20.0.1 - - [10/Mar/2023:11:20:47 +0000] "GET /api/leads/v1/business-entities HTTP/1.1" 500 42 "-" "PostmanRuntime/7.31.1"

The code is as below

function _M.execute(conf)
    local res, ok, err
    local scheme, host, port, path = unpack(http:parse_uri(conf.uam_check_token_api))
    
    local auth_request = _M.new_auth_request(
            host, port, path, token, institution_name, conf.timeout_ms)
    
    local httpc = http.new()
    httpc:set_timeout(conf.timeout_ms)

    if conf.proxy_uri then
        kong.log.notice("Setting the proxy options")
        local proxy_opts = {
            http_proxy = conf.proxy_uri,
            https_proxy = conf.proxy_uri
        }
        httpc:set_proxy_options(proxy_opts)
        kong.log.notice("Setting the proxy options....done")
    end
    kong.log.notice("Scheme : " .. scheme .. " Host : " .. host .. " Port : " .. port)
    local connect_opts = {
        scheme = scheme,
        host = host,
        port = port
    }
    httpc:connect(connect_opts)
    kong.log.notice("Making http request")
    res, err = httpc:request(auth_request)
    kong.log.notice("HTTP call done")
end


function _M.new_auth_request(host, port, path, token, inst, timeout_ms)
    local payload = '{ "sToken": "' .. token }'
    return {
        method = "POST",
        path = path,
        headers = headers,
        body = payload,
        keepalive_timeout = timeout_ms,
        ssl_verify = false
    }
end

[pintsized]

Judging from your error logs, it is crashing on the init phase, not access? You can't use cosockets during the init phase (without a timer light thread).

Again, without full code, I'm left guessing.

Please create a minimal reproducible test case that someone else can try out, and we can go from there. Usually, in doing so, you'll discover what you're doing wrong ;)