Nighthawk_client supports http_proxy and https_proxy

[xiaohui1wang]

Nighthawk_client supports http_proxy and https_proxy.

Assume there is a scenario to use nighthawk_client to test a URL, such as: https://example.com/index.html, but nighthawk_client cannot access "https://example.com/index.html" directly, it needs a proxy server with username and password authentication.
I tried to setup OS environment http_proxy and https_proxy, it doesn't work.
So, is there a way to setup http_proxy and https_proxy for nighthawk_client ? Thanks!

Below is error logs:
./nighthawk_client https://getnighthawk.dev/assets/images/logos/nighthawk/horizontal/nighthawk-text-side.svg
[18:50:40.109348][466408][I] Starting 1 threads / event loops. Time limit: 5 seconds.
[18:50:40.109396][466408][I] Global targets: 100 connections and 5 calls per second.
[18:50:40.879589][466417][E] Exiting due to failing termination predicate
[18:50:40.879612][466417][I] Stopping after 100 ms. Initiated: 1 / Completed: 1. (Completion rate was 9.93226197334181 per second.)
[18:50:41.141247][466408][E] Terminated early because of a failure predicate.
[18:50:41.141269][466408][I] Check the output for problematic counter values. The default Nighthawk failure predicates report failure if (1) Nighthawk could not connect to the target (see 'benchmark.pool_connection_failure' counter; check the address and port number, and try explicitly setting --address-family v4 or v6, especially when using DNS; instead of localhost try 127.0.0.1 or ::1 explicitly), (2) the protocol was not supported by the target (see 'benchmark.stream_resets' counter; check http/https in the URI, --h2), (3) the target returned a 4xx or 5xx HTTP response code (see 'benchmark.http_4xx' and 'benchmark.http_5xx' counters; check the URI path and the server config), or (4) a custom gRPC RequestSource failed. --failure-predicate can be used to relax expectations.
Nighthawk - A layer 7 protocol benchmarking tool.

Initiation to completion (1 samples)
min: 0s 000ms 608us | mean: 0s 000ms 608us | max: 0s 000ms 608us | pstdev: 0s 000ms 000us

Counter Value Per second
benchmark.pool_connection_failure 1 9.93
cluster_manager.cluster_added 1 9.93
default.total_match_count 1 9.93
membership_change 1 9.93
runtime.load_success 1 9.93
runtime.override_dir_not_exists 1 9.93
sequencer.failed_terminations 1 9.93
upstream_cx_connect_fail 1 9.93
upstream_cx_destroy 1 9.93
upstream_cx_destroy_remote 1 9.93
upstream_cx_http1_total 1 9.93
upstream_cx_total 1 9.93
upstream_rq_pending_failure_eject 1 9.93
upstream_rq_pending_total 1 9.93

[18:50:41.147032][466408][E] An error ocurred.

[xiaohui1wang]

Can anyone help to add comments for above question? Thanks!

[eric846]

Nighthawk doesn't currently support http_proxy/https_proxy environment variables or connecting to a proxy using credentials. (I changed this issue from a bug to a question. Feel free to change it to a feature request if the idea below doesn't unblock you.)

• One quick way that you might be able to make progress today is to find a separate third-party HTTP tunnel program and run it on your computer. Disclaimer: I haven't done this myself, so I'm not sure whether there is a trustworthy third-party program that is compatible with your proxy and Nighthawk.
• We would welcome a code contribution if you would like to implement this functionality yourself. Nighthawk is built on an Envoy upstream cluster internally, so there might be simple Envoy settings that Nighthawk could set on the cluster to enable connecting through a proxy.

About the idea of a third-party HTTP tunnel:

Disclaimer: I haven't done this myself, so I'm not sure whether there is a trustworthy third-party program that is compatible with your proxy and Nighthawk.

I'm assuming your goal is to load test a final destination that you can only access through a proxy with credentials.

The tunnel program would:

• connect from your computer to the proxy
• log in to the proxy using your credentials
• ask the proxy to connect to the final destination
• listen for incoming requests on your computer on some port, e.g. 123
• receive incoming requests to localhost:123 and forward them to the proxy
• receive replies from the proxy and return them to the client on localhost:123

Once this tunnel was active and listening on localhost:123, you would run nighthawk_client with http://localhost:123 as its target. Each request from Nighthawk would go through the tunnel program on localhost, to the proxy server, then to the final destination.

If you get something like that working, please let us know!

[eric846]

One "third-party tunnel" you might consider is an Envoy proxy running on your machine between Nighthawk and the password-protected proxy. See envoyproxy/envoy#11308

[eric846]

A couple of things to watch out for:

• The HTTP tunnel might end up being the bottleneck -- may need to keep an eye on it and possibly tune it or give it more resources.
• Either the proxy or the final destination server could be the performance bottleneck. If the bottleneck is not in the component you're trying to test, you might need to adjust the other component, or tune the traffic so the bottleneck is in the right place.

[xiaohui1wang]

Thank you Eric for above response and suggestion.