www.fabiankeil.de/gehacktes/privoxy-tls-benchmarks/
Privoxy supports multiple TLS libraries for HTTPS inspection which lets Privoxy filter encrypted requests and responses.
On this page I'm collecting benchmarks to see how the implementations compare and to make sure Privoxy's performance does not regress.
The test setup currently isn't ideal as I have no dedicated benchmark hardware but it will do for now.
I'm running Privoxy and the system running the benchmarks in bhyve VMs. All systems including the host run ElectroBSD with the ELECTRO_BLOAT kernel configuration.
The host system has two cpu cores:
CPU: Intel(R) Pentium(R) CPU G6950 @ 2.80GHz (2793.05-MHz K8-class CPU) Origin="GenuineIntel" Id=0x20655 Family=0x6 Model=0x25 Stepping=5 Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x82e3bd<SSE3,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,POPCNT> AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM> AMD Features2=0x1<LAHF> VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID TSC: P-state invariant, performance statistics
Note that there's no AESNI support which is unfortunate as some of the TLS libraries could use it.
The VMs are configured to use a single core with 2 GB of memory. The host is also doing other tasks.
I started benchmarking with ElectroBSD 11.4-STABLE and upgraded to ElectroBSD 12.3-STABLE after Privoxy 3.0.33 was released.
Privoxy already supports OpenSSL, LibreSSL and mbedtls and I'm currently working on adding wolfSSL support. The work will be funded using donations made to the Privoxy project. You can donate to the Privoxy project to support this.
The Privoxy versions tested contain experimental patches that haven't been published yet.
This benchmark requests the ElectroBSD homepage reusing connections for multiple requests, thus doing less handshakes. The concurrency level of 1000 results in a fair amount of parallel connections. The connections made by ab-proxy don't seem to be reused fairly, though.
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 4m2.844750331s
Total initiated requests: 30000
Completed requests: 30000
HTTP-200 completed: 29999
HTTP-503 completed: 1
Failed requests: 0
Total transferred: 15306525 bytes
Requests per second: 123.536
Time per request: 8.094825ms
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 3m57.554025599s
Total initiated requests: 30000
Completed requests: 30000
HTTP-200 completed: 30000
Failed requests: 0
Total transferred: 15300000 bytes
Requests per second: 126.287
Time per request: 7.918467ms
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 3m42.1314853s
Total initiated requests: 30000
Completed requests: 30000
HTTP-200 completed: 30000
Failed requests: 0
Total transferred: 15300000 bytes
Requests per second: 135.055
Time per request: 7.404382ms
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 11m0.781294501s
Total initiated requests: 30000
Completed requests: 29997
HTTP-200 completed: 29988
HTTP-502 completed: 5
HTTP-503 completed: 4
Failed requests: 3
Timeout failures: 3
Total transferred: 15360060 bytes
Requests per second: 45.401
Time per request: 22.026043ms
Errors:
3x Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 8m2.315674827s
Total initiated requests: 30000
Completed requests: 29080
HTTP-200 completed: 29080
Failed requests: 920
Timeout failures: 806
Total transferred: 14830800 bytes
Requests per second: 62.200
Time per request: 16.077189ms
Errors:
806x Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
114x Get "https://www.electrobsd.org/": Too many open connections
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 4m37.326142464s
Total initiated requests: 30000
Completed requests: 30000
HTTP-200 completed: 30000
Failed requests: 0
Total transferred: 15300000 bytes
Requests per second: 108.176
Time per request: 9.244204ms
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: keep-alive" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 12m25.87273415s
Total initiated requests: 30000
Completed requests: 28375
HTTP-200 completed: 28375
Failed requests: 1625
Total transferred: 14471250 bytes
Requests per second: 40.221
Time per request: 24.862424ms
Errors:
1625x Get "https://www.electrobsd.org/": Too many open connections
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 3:08 | 692M |
| LibreSSL 3.2.3_2 | 3:12 | 713M |
| LibreSSL devel 3.3.1 | 2:57 | 690M |
| OpenSSL 1.1.1i,1 | 9:50 | 788M |
| OpenSSL devel 3.0.0.a11 | 6:58 | 603M |
| wolfSSL 4.7.0_1 | 3:22 | 828M |
| mbedtls 2.16.9_6 | 8:32 | 510M |
Note that the ElectroBSD.org server does not support TLS 1.3 yet so the libraries that support it can't use it.
| TLS library | Client side TLS version and cipher suite | Server side TLS version and cipher suite |
|---|---|---|
| OpenSSL 1.0.2t | TLSv1.2 AES128-GCM-SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| LibreSSL 3.2.3_2 | TLSv1.3 AEAD-CHACHA20-POLY1305-SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| LibreSSL devel 3.3.1 | TLSv1.3 AEAD-CHACHA20-POLY1305-SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| OpenSSL 1.1.1i,1 | TLSv1.3 TLS_CHACHA20_POLY1305_SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| OpenSSL devel 3.0.0.a11 | TLSv1.3 TLS_CHACHA20_POLY1305_SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| wolfSSL 4.7.0_1 | TLSv1.3 TLS13-AES128-GCM-SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| mbedtls 2.16.9_6 | TLSv1.2 TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256 | TLSv1.2 TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256 |
This benchmark requests the ElectroBSD homepage without reusing connections, thus requiring lots of handshakes.
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 21m16.715068967s
Total initiated requests: 30000
Completed requests: 29719
HTTP-200 completed: 29270
HTTP-503 completed: 449
Failed requests: 281
Timeout failures: 281
Total transferred: 18086415 bytes
Requests per second: 23.498
Time per request: 42.557168ms
Errors:
281x Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 23m27.798118651s
Total initiated requests: 30000
Completed requests: 29668
HTTP-200 completed: 29151
HTTP-503 completed: 517
Failed requests: 332
Timeout failures: 332
Total transferred: 18504105 bytes
Requests per second: 21.310
Time per request: 46.926603ms
Errors:
332x Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 25m4.91063846s
Total initiated requests: 30000
Completed requests: 29672
HTTP-200 completed: 29341
HTTP-503 completed: 331
Failed requests: 328
Timeout failures: 328
Total transferred: 17292495 bytes
Requests per second: 19.935
Time per request: 50.163687ms
Errors:
328x Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 22m0.965955015s
Total initiated requests: 30000
Completed requests: 29355
HTTP-200 completed: 28919
HTTP-503 completed: 436
Failed requests: 645
Timeout failures: 645
Total transferred: 17815950 bytes
Requests per second: 22.711
Time per request: 44.032198ms
Errors:
645x Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 26m55.960774332s
Total initiated requests: 30000
Completed requests: 29871
HTTP-200 completed: 29635
HTTP-503 completed: 236
Failed requests: 129
Timeout failures: 129
Total transferred: 16774110 bytes
Requests per second: 18.565
Time per request: 53.865359ms
Errors:
129x Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 17m46.797803308s
Total initiated requests: 30000
Completed requests: 25512
HTTP-200 completed: 25119
HTTP-503 completed: 393
Failed requests: 4488
Timeout failures: 4488
Total transferred: 15575445 bytes
Requests per second: 28.122
Time per request: 35.559926ms
Errors:
4488x Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1000 -n 10000 --bursts 3 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://www.electrobsd.org/
Benchmarking 'https://www.electrobsd.org/' using proxy 'http://172.16.1.7:8118/' with a total of 30000 GET requests:
Number of bursts: 3
Number of request per burst 10000
Concurrency level: 1000
Time taken for tests: 34m44.465265753s
Total initiated requests: 30000
Completed requests: 29935
HTTP-200 completed: 29789
HTTP-503 completed: 146
Failed requests: 65
Timeout failures: 65
Total transferred: 16219500 bytes
Requests per second: 14.392
Time per request: 69.482175ms
Errors:
65x Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
For this test I added and enabled an invalidate-all-certificates directive that causes Privoxy to generate a new X509 host certificate for each request.
Normally Privoxy generates X509 certificates that are valid for 90 days.
The benchmark is requesting a page from the Privoxy user manual which Privoxy serves itself, therefore there are no outgoing connections. Most of the time is spend generating the 2048 bit RSA key for the certificate.
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 1000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:
Number of bursts: 1
Number of request per burst 1000
Concurrency level: 1
Time taken for tests: 3m51.207656674s
Total initiated requests: 1000
Completed requests: 1000
HTTP-200 completed: 1000
Failed requests: 0
Total transferred: 21587000 bytes
Requests per second: 4.325
Time per request: 231.207656ms
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:
Number of bursts: 1
Number of request per burst 1000
Concurrency level: 1
Time taken for tests: 3m54.853337356s
Total initiated requests: 1000
Completed requests: 1000
HTTP-200 completed: 1000
Failed requests: 0
Total transferred: 21587000 bytes
Requests per second: 4.258
Time per request: 234.853337ms
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:
Number of bursts: 1
Number of request per burst 1000
Concurrency level: 1
Time taken for tests: 4m18.215755982s
Total initiated requests: 1000
Completed requests: 1000
HTTP-200 completed: 1000
Failed requests: 0
Total transferred: 21587000 bytes
Requests per second: 3.873
Time per request: 258.215755ms
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:
Number of bursts: 1
Number of request per burst 1000
Concurrency level: 1
Time taken for tests: 5m59.344970629s
Total initiated requests: 1000
Completed requests: 999
HTTP-200 completed: 999
Failed requests: 1
Timeout failures: 1
Total transferred: 21565413 bytes
Requests per second: 2.783
Time per request: 359.34497ms
Errors:
1x Get "https://p.p/user-manual/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:
Number of bursts: 1
Number of request per burst 1000
Concurrency level: 1
Time taken for tests: 14m8.243950082s
Total initiated requests: 1000
Completed requests: 999
HTTP-200 completed: 999
Failed requests: 1
Timeout failures: 1
Total transferred: 21565413 bytes
Requests per second: 1.179
Time per request: 848.24395ms
Errors:
1x Get "https://p.p/user-manual/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:
Number of bursts: 1
Number of request per burst 1000
Concurrency level: 1
Time taken for tests: 10m6.475444546s
Total initiated requests: 1000
Completed requests: 1000
HTTP-200 completed: 1000
Failed requests: 0
Total transferred: 21587000 bytes
Requests per second: 1.649
Time per request: 606.475444ms
[fk@benchmark-vm ~]$ go/bin/ab-proxy -s 5000 -c 1 -n 1000 --proxy http://172.16.1.7:8118/ --show-errors -H "Connection: close" https://p.p/user-manual/
Benchmarking 'https://p.p/user-manual/' using proxy 'http://172.16.1.7:8118/' with a total of 1000 GET requests:
Number of bursts: 1
Number of request per burst 1000
Concurrency level: 1
Time taken for tests: 6m7.916393792s
Total initiated requests: 1000
Completed requests: 1000
HTTP-200 completed: 1000
Failed requests: 0
Total transferred: 21587000 bytes
Requests per second: 2.718
Time per request: 367.916393ms
I have not yet properly investigated the ab-proxy errors.
It's suspicious that proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
errors were reported even for some of the RSA key generation tests
with concurrency level 1.
It's not obvious to me which limit is being hit when ab-proxy complaints about
Too many open connections
.
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 70.8044 secs Slowest: 39.0221 secs Fastest: 0.0100 secs Average: 6.5599 secs Requests/sec: 141.2341 Response time histogram: 0.010 [1] | 3.911 [4166] |________________________________________ 7.812 [3032] |_____________________________ 11.714 [1363] |_____________ 15.615 [633] |______ 19.516 [329] |___ 23.417 [269] |___ 27.318 [87] |_ 31.220 [38] | 35.121 [61] |_ 39.022 [21] | Latency distribution: 10% in 2.3938 secs 25% in 3.1952 secs 50% in 4.2323 secs 75% in 8.4616 secs 90% in 13.7324 secs 95% in 18.9752 secs 99% in 29.2693 secs Details (average, fastest, slowest): DNS+dialup: 0.1407 secs, 0.0100 secs, 39.0221 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0005 secs, 0.0000 secs, 0.1165 secs resp wait: 1.6825 secs, 0.0080 secs, 27.2391 secs resp read: 0.0021 secs, 0.0001 secs, 0.5911 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 58.0847 secs Slowest: 29.9009 secs Fastest: 0.0104 secs Average: 5.1935 secs Requests/sec: 172.1623 Response time histogram: 0.010 [1] | 2.999 [1384] |________ 5.989 [6588] |________________________________________ 8.978 [770] |_____ 11.967 [436] |___ 14.956 [333] |__ 17.945 [220] |_ 20.934 [183] |_ 23.923 [43] | 26.912 [27] | 29.901 [15] | Latency distribution: 10% in 2.7917 secs 25% in 3.1824 secs 50% in 3.5952 secs 75% in 4.7549 secs 90% in 10.4189 secs 95% in 14.8431 secs 99% in 20.5910 secs Details (average, fastest, slowest): DNS+dialup: 0.0543 secs, 0.0104 secs, 29.9009 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0005 secs, 0.0000 secs, 0.5658 secs resp wait: 0.8956 secs, 0.0082 secs, 26.5739 secs resp read: 0.0018 secs, 0.0001 secs, 0.6613 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 45.4728 secs Slowest: 45.1093 secs Fastest: 0.0089 secs Average: 3.9195 secs Requests/sec: 219.9116 Response time histogram: 0.009 [1] | 4.519 [8462] |________________________________________ 9.029 [1172] |______ 13.539 [234] |_ 18.049 [3] | 22.559 [9] | 27.069 [6] | 31.579 [17] | 36.089 [13] | 40.599 [8] | 45.109 [75] | Latency distribution: 10% in 0.7545 secs 25% in 2.8709 secs 50% in 3.3514 secs 75% in 3.8533 secs 90% in 5.9146 secs 95% in 8.7582 secs 99% in 30.5981 secs Details (average, fastest, slowest): DNS+dialup: 0.0396 secs, 0.0089 secs, 45.1093 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0003 secs, 0.0000 secs, 0.0647 secs resp wait: 0.7306 secs, 0.0079 secs, 44.5228 secs resp read: 0.0024 secs, 0.0001 secs, 8.5983 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 53.5002 secs Slowest: 53.2844 secs Fastest: 0.0084 secs Average: 4.6653 secs Requests/sec: 186.9150 Response time histogram: 0.008 [1] | 5.336 [7242] |________________________________________ 10.664 [2562] |______________ 15.991 [111] |_ 21.319 [25] | 26.646 [3] | 31.974 [1] | 37.302 [2] | 42.629 [0] | 47.957 [8] | 53.284 [45] | Latency distribution: 10% in 2.6186 secs 25% in 3.1237 secs 50% in 4.2889 secs 75% in 5.5426 secs 90% in 7.2864 secs 95% in 8.0298 secs 99% in 14.1800 secs Details (average, fastest, slowest): DNS+dialup: 0.0127 secs, 0.0084 secs, 53.2844 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0003 secs, 0.0000 secs, 0.0691 secs resp wait: 0.5642 secs, 0.0082 secs, 52.4999 secs resp read: 0.0018 secs, 0.0001 secs, 0.8801 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 45.2568 secs Slowest: 44.3002 secs Fastest: 0.0109 secs Average: 4.1396 secs Requests/sec: 220.9610 Response time histogram: 0.011 [1] | 4.440 [7903] |________________________________________ 8.869 [1985] |__________ 13.298 [14] | 17.727 [23] | 22.156 [25] | 26.584 [11] | 31.013 [10] | 35.442 [8] | 39.871 [14] | 44.300 [6] | Latency distribution: 10% in 3.0057 secs 25% in 3.4220 secs 50% in 3.8525 secs 75% in 4.3496 secs 90% in 5.6387 secs 95% in 7.0523 secs 99% in 12.3051 secs Details (average, fastest, slowest): DNS+dialup: 0.0347 secs, 0.0109 secs, 44.3002 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0007 secs, 0.0000 secs, 0.4245 secs resp wait: 0.4287 secs, 0.0082 secs, 41.7396 secs resp read: 0.0023 secs, 0.0001 secs, 2.8815 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 70.0007 secs Slowest: 41.1610 secs Fastest: 0.0085 secs Average: 5.8105 secs Requests/sec: 142.8557 Response time histogram: 0.009 [1] | 4.124 [5269] |________________________________________ 8.239 [3145] |________________________ 12.354 [695] |_____ 16.470 [348] |___ 20.585 [117] |_ 24.700 [225] |__ 28.815 [19] | 32.931 [42] | 37.046 [31] | 41.161 [108] |_ Latency distribution: 10% in 2.0746 secs 25% in 3.2209 secs 50% in 4.0434 secs 75% in 5.4152 secs 90% in 11.4100 secs 95% in 17.8118 secs 99% in 37.3581 secs Details (average, fastest, slowest): DNS+dialup: 0.0093 secs, 0.0085 secs, 41.1610 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0004 secs, 0.0000 secs, 0.4865 secs resp wait: 1.2999 secs, 0.0079 secs, 34.2348 secs resp read: 0.0021 secs, 0.0001 secs, 2.0635 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 369.2357 secs Slowest: 166.0667 secs Fastest: 0.0091 secs Average: 30.3497 secs Requests/sec: 27.0830 Response time histogram: 0.009 [1] | 16.615 [6120] |________________________________________ 33.221 [185] |_ 49.826 [575] |____ 66.432 [413] |___ 83.038 [170] |_ 99.644 [173] |_ 116.249 [266] |__ 132.855 [630] |____ 149.461 [192] |_ 166.067 [192] |_ Latency distribution: 10% in 0.9116 secs 25% in 2.9378 secs 50% in 4.5298 secs 75% in 44.1925 secs 90% in 121.4544 secs 95% in 130.6333 secs 99% in 155.0877 secs Details (average, fastest, slowest): DNS+dialup: 0.9649 secs, 0.0091 secs, 166.0667 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.2049 secs, 0.0000 secs, 27.5575 secs resp wait: 14.5920 secs, 0.0086 secs, 111.2285 secs resp read: 0.2987 secs, 0.0001 secs, 25.8274 secs Status code distribution: [200] 8917 responses Error distribution: [1083] Get "https://www.electrobsd.org/": context deadline exceeded
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 586.4203 secs Slowest: 385.5857 secs Fastest: 0.0861 secs Average: 37.4564 secs Requests/sec: 17.0526 Total data: 429135 bytes Size/request: 45 bytes Response time histogram: 0.086 [1] | 38.636 [6089] |________________________________________ 77.186 [3258] |_____________________ 115.736 [110] |_ 154.286 [1] | 192.836 [0] | 231.386 [0] | 269.936 [0] | 308.486 [0] | 347.036 [0] | 385.586 [20] | Latency distribution: 10% in 25.5035 secs 25% in 29.9598 secs 50% in 34.5757 secs 75% in 43.7889 secs 90% in 50.2532 secs 95% in 62.4555 secs 99% in 89.7409 secs Details (average, fastest, slowest): DNS+dialup: 29.9351 secs, 0.0861 secs, 385.5857 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0005 secs, 0.0000 secs, 0.2965 secs resp wait: 7.5154 secs, 0.0719 secs, 362.7325 secs resp read: 0.0053 secs, 0.0001 secs, 6.6434 secs Status code distribution: [200] 9418 responses [503] 61 responses Error distribution: [363] Get "https://www.electrobsd.org/": context deadline exceeded [158] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 658.7700 secs Slowest: 358.0393 secs Fastest: 0.0796 secs Average: 44.1614 secs Requests/sec: 15.1798 Total data: 729126 bytes Size/request: 75 bytes Response time histogram: 0.080 [1] | 35.876 [1482] |________ 71.672 [7671] |________________________________________ 107.468 [371] |__ 143.263 [58] | 179.059 [1] | 214.855 [0] | 250.651 [0] | 286.447 [0] | 322.243 [1] | 358.039 [9] | Latency distribution: 10% in 34.0873 secs 25% in 37.6683 secs 50% in 41.8934 secs 75% in 47.3630 secs 90% in 59.8096 secs 95% in 69.1311 secs 99% in 93.7552 secs Details (average, fastest, slowest): DNS+dialup: 37.2611 secs, 0.0796 secs, 358.0393 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.0114 secs resp wait: 6.8963 secs, 0.0701 secs, 330.3939 secs resp read: 0.0038 secs, 0.0001 secs, 7.4791 secs Status code distribution: [200] 9495 responses [502] 57 responses [503] 42 responses Error distribution: [364] Get "https://www.electrobsd.org/": context deadline exceeded [42] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 498.4878 secs Slowest: 123.9539 secs Fastest: 0.0959 secs Average: 44.9095 secs Requests/sec: 20.0607 Total data: 1020075 bytes Size/request: 102 bytes Response time histogram: 0.096 [1] | 12.482 [275] |__ 24.868 [100] |_ 37.253 [1863] |_________________ 49.639 [4511] |________________________________________ 62.025 [2244] |____________________ 74.411 [869] |________ 86.797 [53] | 99.182 [14] | 111.568 [10] | 123.954 [1] | Latency distribution: 10% in 32.2111 secs 25% in 37.8849 secs 50% in 43.8031 secs 75% in 52.6480 secs 90% in 61.1590 secs 95% in 67.6880 secs 99% in 73.4740 secs Details (average, fastest, slowest): DNS+dialup: 38.6913 secs, 0.0959 secs, 123.9539 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0003 secs, 0.0000 secs, 0.0171 secs resp wait: 6.2129 secs, 0.0804 secs, 52.7336 secs resp read: 0.0050 secs, 0.0001 secs, 10.3844 secs Status code distribution: [200] 9796 responses [503] 145 responses Error distribution: [54] Get "https://www.electrobsd.org/": context deadline exceeded [5] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 448.7498 secs Slowest: 95.4216 secs Fastest: 0.1897 secs Average: 40.1372 secs Requests/sec: 22.2841 Total data: 527625 bytes Size/request: 53 bytes Response time histogram: 0.190 [1] | 9.713 [116] |_ 19.236 [375] |___ 28.759 [120] |_ 38.282 [5068] |________________________________________ 47.806 [1613] |_____________ 57.329 [1932] |_______________ 66.852 [672] |_____ 76.375 [23] | 85.898 [25] | 95.422 [8] | Latency distribution: 10% in 31.4769 secs 25% in 34.1200 secs 50% in 37.1113 secs 75% in 49.0653 secs 90% in 55.7705 secs 95% in 58.4753 secs 99% in 64.5587 secs Details (average, fastest, slowest): DNS+dialup: 35.7682 secs, 0.1897 secs, 95.4216 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0003 secs, 0.0000 secs, 0.1436 secs resp wait: 4.3613 secs, 0.0876 secs, 51.5034 secs resp read: 0.0074 secs, 0.0001 secs, 5.0321 secs Status code distribution: [200] 9878 responses [503] 75 responses Error distribution: [47] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 540.4711 secs Slowest: 107.6423 secs Fastest: 1.5339 secs Average: 48.4370 secs Requests/sec: 18.5024 Total data: 196980 bytes Size/request: 20 bytes Response time histogram: 1.534 [1] | 12.145 [12] | 22.756 [46] | 33.366 [217] |__ 43.977 [2850] |_____________________________ 54.588 [3893] |________________________________________ 65.199 [2472] |_________________________ 75.810 [175] |__ 86.421 [17] | 97.031 [2] | 107.642 [1] | Latency distribution: 10% in 39.5666 secs 25% in 42.9131 secs 50% in 46.9803 secs 75% in 55.1250 secs 90% in 58.6916 secs 95% in 60.9301 secs 99% in 70.8738 secs Details (average, fastest, slowest): DNS+dialup: 43.6338 secs, 1.5339 secs, 107.6423 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.0225 secs resp wait: 4.8004 secs, 0.0896 secs, 43.6299 secs resp read: 0.0026 secs, 0.0001 secs, 1.1191 secs Status code distribution: [200] 9658 responses [503] 28 responses Error distribution: [275] Get "https://www.electrobsd.org/": context deadline exceeded [39] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/
Summary:
Total: 651.9580 secs
Slowest: 356.3045 secs
Fastest: 0.0758 secs
Average: 31.9959 secs
Requests/sec: 15.3384
Total data: 316575 bytes
Size/request: 31 bytes
Response time histogram:
0.076 [1] |
35.699 [7634] |________________________________________
71.322 [2303] |____________
106.944 [9] |
142.567 [0] |
178.190 [0] |
213.813 [0] |
249.436 [0] |
285.059 [0] |
320.682 [0] |
356.304 [4] |
Latency distribution:
10% in 24.4374 secs
25% in 27.4192 secs
50% in 30.1033 secs
75% in 34.9318 secs
90% in 48.9801 secs
95% in 52.6747 secs
99% in 56.6523 secs
Details (average, fastest, slowest):
DNS+dialup: 26.3534 secs, 0.0758 secs, 356.3045 secs
DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs
req write: 0.0007 secs, 0.0000 secs, 0.1846 secs
resp wait: 5.6347 secs, 0.0587 secs, 331.2605 secs
resp read: 0.0071 secs, 0.0001 secs, 2.1032 secs
Status code distribution:
[200] 9902 responses
[403] 4 responses
[503] 45 responses
Error distribution:
[49] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
The 403 errors are the result of the certificate validation failing.
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 745.4701 secs Slowest: 132.6298 secs Fastest: 0.1535 secs Average: 68.3000 secs Requests/sec: 13.4144 Total data: 239190 bytes Size/request: 23 bytes Response time histogram: 0.154 [1] | 13.401 [39] |_ 26.649 [137] |__ 39.896 [885] |____________ 53.144 [1675] |________________________ 66.392 [1406] |____________________ 79.639 [2068] |_____________________________ 92.887 [2845] |________________________________________ 106.135 [830] |____________ 119.382 [102] |_ 132.630 [2] | Latency distribution: 10% in 39.3582 secs 25% in 51.0014 secs 50% in 72.5942 secs 75% in 84.0988 secs 90% in 92.1742 secs 95% in 96.1419 secs 99% in 106.3705 secs Details (average, fastest, slowest): DNS+dialup: 58.9630 secs, 0.1535 secs, 132.6298 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.4718 secs, 0.0000 secs, 9.6484 secs resp wait: 7.9882 secs, 0.0836 secs, 58.6134 secs resp read: 0.8770 secs, 0.0002 secs, 18.9068 secs Status code distribution: [200] 9956 responses [503] 34 responses Error distribution: [10] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 5:56 | 504M |
| LibreSSL 3.2.3_2 | 6:35 | 586M |
| LibreSSL devel 3.3.1 | 6:55 | 503M |
| OpenSSL 1.1.1i,1 | 6:12 | 310M |
| OpenSSL devel 3.0.0.a11 | 7:30 | 385M |
| wolfSSL 4.7.0_1 | 4:37 | 302M |
| mbedtls 2.16.9_6 | 8:17 | 304M |
| TLS library | Client side TLS version and cipher suite | Server side TLS version and cipher suite |
|---|---|---|
| OpenSSL 1.0.2t | TLSv1.2 AES128-GCM-SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| LibreSSL 3.2.3_2 | TLSv1.3 AEAD-CHACHA20-POLY1305-SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| LibreSSL devel 3.3.1 | TLSv1.3 AEAD-CHACHA20-POLY1305-SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| OpenSSL 1.1.1i,1 | TLSv1.3 TLS_CHACHA20_POLY1305_SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| OpenSSL devel 3.0.0.a11 | TLSv1.3 TLS_CHACHA20_POLY1305_SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| wolfSSL 4.7.0_1 | TLSv1.3 TLS13-AES128-GCM-SHA256 | TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256 |
| mbedtls 2.16.9_6 | TLSv1.2 TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256 | TLSv1.2 TLS-ECDHE-RSA-WITH-AES-128-GCM-SHA256 |
I have not yet properly investigated the hey errors either. The -t option obviously does not prevent timeouts from occurring.
The response time histograms are probably worth investigating as well.
The OpenSSL port has been updated to 1.1.1j. wolfSSL has been recompiled with --enable-sp-asm and --enable-sp-math-all added to the configure flags which significantly speeds up RSA key generation (and probably other things as well). privoxy-experimental-ports-openssl 3.0.32.20210216 and privoxy-experimental-wolfssl 3.0.32.20210216 have been recompiled, the other ports remain the same.
Privoxy configured as described above to force RSA key regeneration.
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/ Summary: Total: 205.3925 secs Slowest: 6.3301 secs Fastest: 0.0301 secs Average: 0.2054 secs Requests/sec: 4.8687 Total data: 21587000 bytes Size/request: 21587 bytes Response time histogram: 0.030 [1] | 0.660 [997] |________________________________________ 1.290 [1] | 1.920 [0] | 2.550 [0] | 3.180 [0] | 3.810 [0] | 4.440 [0] | 5.070 [0] | 5.700 [0] | 6.330 [1] | Latency distribution: 10% in 0.0731 secs 25% in 0.1124 secs 50% in 0.1740 secs 75% in 0.2601 secs 90% in 0.3597 secs 95% in 0.4293 secs 99% in 0.5754 secs Details (average, fastest, slowest): DNS+dialup: 0.2021 secs, 0.0301 secs, 6.3301 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0001 secs, 0.0001 secs, 0.0002 secs resp wait: 0.0012 secs, 0.0009 secs, 0.0857 secs resp read: 0.0019 secs, 0.0011 secs, 0.0655 secs Status code distribution: [200] 1000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/ Summary: Total: 285.2183 secs Slowest: 0.8314 secs Fastest: 0.0340 secs Average: 0.2104 secs Requests/sec: 3.5061 Total data: 21565413 bytes Size/request: 21587 bytes Response time histogram: 0.034 [1] | 0.114 [226] |____________________________ 0.193 [321] |________________________________________ 0.273 [193] |________________________ 0.353 [120] |_______________ 0.433 [72] |_________ 0.512 [36] |____ 0.592 [20] |__ 0.672 [6] |_ 0.752 [1] | 0.831 [3] | Latency distribution: 10% in 0.0806 secs 25% in 0.1185 secs 50% in 0.1793 secs 75% in 0.2768 secs 90% in 0.3825 secs 95% in 0.4715 secs 99% in 0.6071 secs Details (average, fastest, slowest): DNS+dialup: 0.2078 secs, 0.0340 secs, 0.8314 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0001 secs, 0.0001 secs, 0.0002 secs resp wait: 0.0011 secs, 0.0009 secs, 0.0167 secs resp read: 0.0014 secs, 0.0003 secs, 0.0145 secs Status code distribution: [200] 999 responses Error distribution: [1] Get "https://p.p/user-manual/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/ Summary: Total: 412.1682 secs Slowest: 59.0361 secs Fastest: 0.0310 secs Average: 0.2627 secs Requests/sec: 2.4262 Total data: 21543826 bytes Size/request: 21587 bytes Response time histogram: 0.031 [1] | 5.932 [996] |________________________________________ 11.832 [0] | 17.733 [0] | 23.633 [0] | 29.534 [0] | 35.434 [0] | 41.335 [0] | 47.235 [0] | 53.136 [0] | 59.036 [1] | Latency distribution: 10% in 0.0743 secs 25% in 0.1081 secs 50% in 0.1772 secs 75% in 0.2612 secs 90% in 0.3695 secs 95% in 0.4542 secs 99% in 0.7016 secs Details (average, fastest, slowest): DNS+dialup: 0.2583 secs, 0.0310 secs, 59.0361 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0001 secs, 0.0001 secs, 0.0003 secs resp wait: 0.0029 secs, 0.0009 secs, 0.1069 secs resp read: 0.0014 secs, 0.0003 secs, 0.0221 secs Status code distribution: [200] 998 responses Error distribution: [2] Get "https://p.p/user-manual/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/ Summary: Total: 331.6408 secs Slowest: 34.4816 secs Fastest: 0.0321 secs Average: 0.2569 secs Requests/sec: 3.0153 Total data: 21565413 bytes Size/request: 21587 bytes Response time histogram: 0.032 [1] | 3.477 [996] |________________________________________ 6.922 [0] | 10.367 [0] | 13.812 [0] | 17.257 [0] | 20.702 [0] | 24.147 [1] | 27.592 [0] | 31.037 [0] | 34.482 [1] | Latency distribution: 10% in 0.0746 secs 25% in 0.1114 secs 50% in 0.1692 secs 75% in 0.2572 secs 90% in 0.3718 secs 95% in 0.4698 secs 99% in 0.6620 secs Details (average, fastest, slowest): DNS+dialup: 0.2529 secs, 0.0321 secs, 34.4816 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0001 secs, 0.0001 secs, 0.0002 secs resp wait: 0.0025 secs, 0.0009 secs, 0.9218 secs resp read: 0.0014 secs, 0.0003 secs, 0.0318 secs Status code distribution: [200] 999 responses Error distribution: [1] Get "https://p.p/user-manual/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/ Summary: Total: 694.1342 secs Slowest: 2.3307 secs Fastest: 0.1918 secs Average: 0.6941 secs Requests/sec: 1.4406 Total data: 21587000 bytes Size/request: 21587 bytes Response time histogram: 0.192 [1] | 0.406 [219] |_____________________________ 0.620 [306] |________________________________________ 0.833 [187] |________________________ 1.047 [146] |___________________ 1.261 [60] |________ 1.475 [44] |______ 1.689 [16] |__ 1.903 [10] |_ 2.117 [8] |_ 2.331 [3] | Latency distribution: 10% in 0.3290 secs 25% in 0.4316 secs 50% in 0.5993 secs 75% in 0.8770 secs 90% in 1.1933 secs 95% in 1.3963 secs 99% in 1.9116 secs Details (average, fastest, slowest): DNS+dialup: 0.6905 secs, 0.1918 secs, 2.3307 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0001 secs, 0.0001 secs, 0.0002 secs resp wait: 0.0018 secs, 0.0010 secs, 0.0243 secs resp read: 0.0018 secs, 0.0002 secs, 0.0284 secs Status code distribution: [200] 1000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/ Summary: Total: 158.4484 secs Slowest: 1.3990 secs Fastest: 0.0402 secs Average: 0.1584 secs Requests/sec: 6.3112 Total data: 21587000 bytes Size/request: 21587 bytes Response time histogram: 0.040 [1] | 0.176 [685] |________________________________________ 0.312 [260] |_______________ 0.448 [43] |___ 0.584 [10] |_ 0.720 [0] | 0.855 [0] | 0.991 [0] | 1.127 [0] | 1.263 [0] | 1.399 [1] | Latency distribution: 10% in 0.0706 secs 25% in 0.0961 secs 50% in 0.1392 secs 75% in 0.1981 secs 90% in 0.2687 secs 95% in 0.3204 secs 99% in 0.4758 secs Details (average, fastest, slowest): DNS+dialup: 0.1551 secs, 0.0402 secs, 1.3990 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0001 secs, 0.0001 secs, 0.0008 secs resp wait: 0.0013 secs, 0.0009 secs, 0.0464 secs resp read: 0.0018 secs, 0.0006 secs, 0.0230 secs Status code distribution: [200] 1000 responses
Clearly the configure flag changes for wolfSSL paid off.
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 1000 -c 1 -t 0 -disable-keepalive https://p.p/user-manual/ Summary: Total: 320.8540 secs Slowest: 2.2626 secs Fastest: 0.1000 secs Average: 0.3208 secs Requests/sec: 3.1167 Total data: 21587000 bytes Size/request: 21587 bytes Response time histogram: 0.100 [1] | 0.316 [589] |________________________________________ 0.532 [314] |_____________________ 0.749 [72] |_____ 0.965 [21] |_ 1.181 [2] | 1.398 [0] | 1.614 [0] | 1.830 [0] | 2.046 [0] | 2.263 [1] | Latency distribution: 10% in 0.1527 secs 25% in 0.2001 secs 50% in 0.2826 secs 75% in 0.4036 secs 90% in 0.5273 secs 95% in 0.6218 secs 99% in 0.8662 secs Details (average, fastest, slowest): DNS+dialup: 0.3149 secs, 0.1000 secs, 2.2626 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0009 secs, 0.0001 secs, 0.0431 secs resp wait: 0.0019 secs, 0.0009 secs, 0.1673 secs resp read: 0.0029 secs, 0.0002 secs, 0.1259 secs Status code distribution: [200] 1000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 142.3040 secs Slowest: 60.2315 secs Fastest: 0.0119 secs Average: 9.2436 secs Requests/sec: 70.2721 Response time histogram: 0.012 [1] | 6.034 [3364] |____________________________ 12.056 [4818] |________________________________________ 18.078 [1301] |___________ 24.100 [217] |__ 30.122 [55] | 36.144 [10] | 42.166 [1] | 48.188 [0] | 54.210 [51] | 60.232 [182] |__ Latency distribution: 10% in 3.5249 secs 25% in 4.1903 secs 50% in 7.4471 secs 75% in 10.2701 secs 90% in 17.0103 secs 95% in 18.5290 secs 99% in 54.6801 secs Details (average, fastest, slowest): DNS+dialup: 0.0981 secs, 0.0119 secs, 60.2315 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.1230 secs resp wait: 1.1306 secs, 0.0082 secs, 24.3279 secs resp read: 0.0019 secs, 0.0001 secs, 5.1493 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 42.1387 secs Slowest: 41.8936 secs Fastest: 0.0109 secs Average: 3.8552 secs Requests/sec: 237.3115 Response time histogram: 0.011 [1] | 4.199 [8660] |________________________________________ 8.387 [716] |___ 12.576 [548] |___ 16.764 [15] | 20.952 [5] | 25.141 [24] | 29.329 [2] | 33.517 [3] | 37.705 [11] | 41.894 [15] | Latency distribution: 10% in 2.8191 secs 25% in 3.0436 secs 50% in 3.3098 secs 75% in 3.5999 secs 90% in 5.9010 secs 95% in 9.0816 secs 99% in 10.1696 secs Details (average, fastest, slowest): DNS+dialup: 0.0110 secs, 0.0109 secs, 41.8936 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.1999 secs resp wait: 0.3520 secs, 0.0083 secs, 41.0545 secs resp read: 0.0016 secs, 0.0001 secs, 0.1989 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 54.8621 secs Slowest: 35.7601 secs Fastest: 0.0175 secs Average: 4.5901 secs Requests/sec: 182.2751 Response time histogram: 0.018 [1] | 3.592 [2239] |_____________ 7.166 [6899] |________________________________________ 10.740 [669] |____ 14.315 [41] | 17.889 [88] |_ 21.463 [16] | 25.037 [18] | 28.612 [10] | 32.186 [13] | 35.760 [6] | Latency distribution: 10% in 3.1132 secs 25% in 3.6593 secs 50% in 3.9741 secs 75% in 4.8641 secs 90% in 6.8332 secs 95% in 8.6111 secs 99% in 16.4391 secs Details (average, fastest, slowest): DNS+dialup: 0.0245 secs, 0.0175 secs, 35.7601 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0003 secs, 0.0000 secs, 0.2310 secs resp wait: 0.4633 secs, 0.0083 secs, 34.4497 secs resp read: 0.0018 secs, 0.0001 secs, 0.3882 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 54.5644 secs Slowest: 25.7727 secs Fastest: 0.0150 secs Average: 4.7654 secs Requests/sec: 183.2696 Response time histogram: 0.015 [1] | 2.591 [975] |______ 5.167 [6311] |________________________________________ 7.742 [1544] |__________ 10.318 [768] |_____ 12.894 [222] |_ 15.470 [51] | 18.045 [50] | 20.621 [32] | 23.197 [23] | 25.773 [23] | Latency distribution: 10% in 2.6062 secs 25% in 3.2676 secs 50% in 3.8216 secs 75% in 5.3538 secs 90% in 8.1412 secs 95% in 10.0502 secs 99% in 17.0874 secs Details (average, fastest, slowest): DNS+dialup: 0.0199 secs, 0.0150 secs, 25.7727 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0004 secs, 0.0000 secs, 0.4120 secs resp wait: 0.4434 secs, 0.0082 secs, 24.1244 secs resp read: 0.0016 secs, 0.0001 secs, 0.4653 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 43.0776 secs Slowest: 32.3190 secs Fastest: 0.0118 secs Average: 4.0326 secs Requests/sec: 232.1393 Response time histogram: 0.012 [1] | 3.243 [2945] |_________________ 6.473 [6818] |________________________________________ 9.704 [135] |_ 12.935 [24] | 16.165 [20] | 19.396 [11] | 22.627 [24] | 25.858 [10] | 29.088 [7] | 32.319 [5] | Latency distribution: 10% in 2.9814 secs 25% in 3.1775 secs 50% in 3.7742 secs 75% in 4.6033 secs 90% in 5.5741 secs 95% in 6.0325 secs 99% in 9.7961 secs Details (average, fastest, slowest): DNS+dialup: 0.0092 secs, 0.0118 secs, 32.3190 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0003 secs, 0.0000 secs, 0.4390 secs resp wait: 0.3160 secs, 0.0082 secs, 31.6838 secs resp read: 0.0025 secs, 0.0001 secs, 9.0801 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 78.1368 secs Slowest: 49.5334 secs Fastest: 0.0125 secs Average: 7.0439 secs Requests/sec: 127.9807 Response time histogram: 0.012 [1] | 4.965 [5003] |________________________________________ 9.917 [2875] |_______________________ 14.869 [793] |______ 19.821 [900] |_______ 24.773 [80] |_ 29.725 [161] |_ 34.677 [57] | 39.629 [28] | 44.581 [21] | 49.533 [81] |_ Latency distribution: 10% in 1.6025 secs 25% in 2.5724 secs 50% in 4.9548 secs 75% in 7.9658 secs 90% in 16.5569 secs 95% in 19.1906 secs 99% in 39.6885 secs Details (average, fastest, slowest): DNS+dialup: 0.0878 secs, 0.0125 secs, 49.5334 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.0688 secs resp wait: 1.5907 secs, 0.0081 secs, 46.9209 secs resp read: 0.0019 secs, 0.0001 secs, 1.6332 secs Status code distribution: [200] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 https://www.electrobsd.org/ Summary: Total: 387.9665 secs Slowest: 291.5755 secs Fastest: 0.0087 secs Average: 34.6771 secs Requests/sec: 25.7754 Total data: 899589 bytes Size/request: 99 bytes Response time histogram: 0.009 [1] | 29.165 [6664] |________________________________________ 58.322 [549] |___ 87.479 [727] |____ 116.635 [160] |_ 145.792 [153] |_ 174.949 [79] | 204.105 [162] |_ 233.262 [506] |___ 262.419 [52] | 291.575 [18] | Latency distribution: 10% in 0.5065 secs 25% in 2.3675 secs 50% in 4.2593 secs 75% in 34.5339 secs 90% in 127.3195 secs 95% in 215.5636 secs 99% in 229.7423 secs Details (average, fastest, slowest): DNS+dialup: 4.1781 secs, 0.0087 secs, 291.5755 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.1664 secs, 0.0000 secs, 30.5812 secs resp wait: 19.2848 secs, 0.0083 secs, 206.5032 secs resp read: 0.0796 secs, 0.0001 secs, 19.7582 secs Status code distribution: [200] 8945 responses [502] 23 responses [503] 103 responses Error distribution: [929] Get "https://www.electrobsd.org/": context deadline exceeded
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 1:33 | 478M |
| LibreSSL 3.2.3_2 | 1:07 | 415M |
| LibreSSL devel 3.3.1 | 1:17 | 452M |
| OpenSSL 1.1.1j,1 | 1:13 | 562M |
| OpenSSL devel 3.0.0.a11 | 2:12 | 432M |
| wolfSSL 4.7.0_2 | 1:39 | 704M |
| mbedtls 2.16.9_6 | 5:15 | 800M |
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 462.4011 secs Slowest: 100.0522 secs Fastest: 0.1048 secs Average: 41.1673 secs Requests/sec: 21.6262 Total data: 204015 bytes Size/request: 20 bytes Response time histogram: 0.105 [1] | 10.100 [153] |__ 20.094 [99] |_ 30.089 [769] |________ 40.084 [3313] |____________________________________ 50.078 [3722] |________________________________________ 60.073 [1339] |______________ 70.068 [293] |___ 80.063 [64] |_ 90.057 [5] | 100.052 [2] | Latency distribution: 10% in 29.9606 secs 25% in 34.2167 secs 50% in 41.5538 secs 75% in 47.0471 secs 90% in 54.9308 secs 95% in 59.2613 secs 99% in 65.2045 secs Details (average, fastest, slowest): DNS+dialup: 35.6897 secs, 0.1048 secs, 100.0522 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.0416 secs resp wait: 5.4755 secs, 0.0875 secs, 58.7199 secs resp read: 0.0020 secs, 0.0001 secs, 0.4180 secs Status code distribution: [200] 9731 responses [503] 29 responses Error distribution: [167] Get "https://www.electrobsd.org/": context deadline exceeded [73] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 463.9217 secs Slowest: 109.9825 secs Fastest: 0.0975 secs Average: 41.0221 secs Requests/sec: 21.5554 Total data: 225120 bytes Size/request: 23 bytes Response time histogram: 0.097 [1] | 11.086 [229] |__ 22.075 [346] |___ 33.063 [604] |____ 44.052 [5508] |________________________________________ 55.040 [1948] |______________ 66.029 [858] |______ 77.017 [145] |_ 88.006 [8] | 98.994 [21] | 109.983 [1] | Latency distribution: 10% in 31.8813 secs 25% in 36.1928 secs 50% in 39.0768 secs 75% in 48.2456 secs 90% in 55.4362 secs 95% in 60.2266 secs 99% in 69.5408 secs Details (average, fastest, slowest): DNS+dialup: 36.5161 secs, 0.0975 secs, 109.9825 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.1034 secs resp wait: 4.5031 secs, 0.0834 secs, 62.3988 secs resp read: 0.0026 secs, 0.0001 secs, 0.7376 secs Status code distribution: [200] 9637 responses [503] 32 responses Error distribution: [271] Get "https://www.electrobsd.org/": context deadline exceeded [60] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 494.6845 secs Slowest: 114.8360 secs Fastest: 0.1857 secs Average: 44.0541 secs Requests/sec: 20.2149 Total data: 710535 bytes Size/request: 73 bytes Response time histogram: 0.186 [1] | 11.651 [253] |__ 23.116 [51] | 34.581 [1259] |__________ 46.046 [5174] |________________________________________ 57.511 [1329] |__________ 68.976 [1183] |_________ 80.441 [102] |_ 91.906 [160] |_ 103.371 [121] |_ 114.836 [9] | Latency distribution: 10% in 32.7068 secs 25% in 36.2907 secs 50% in 40.5670 secs 75% in 50.4742 secs 90% in 62.2500 secs 95% in 66.2989 secs 99% in 95.0129 secs Details (average, fastest, slowest): DNS+dialup: 38.1360 secs, 0.1857 secs, 114.8360 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0004 secs, 0.0000 secs, 0.1633 secs resp wait: 5.9142 secs, 0.0813 secs, 62.8772 secs resp read: 0.0035 secs, 0.0001 secs, 0.6897 secs Status code distribution: [200] 9541 responses [503] 101 responses Error distribution: [303] Get "https://www.electrobsd.org/": context deadline exceeded [55] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 532.5850 secs Slowest: 129.8773 secs Fastest: 0.1238 secs Average: 47.0465 secs Requests/sec: 18.7763 Total data: 7260120 bytes Size/request: 729 bytes Response time histogram: 0.124 [1] | 13.099 [264] |___ 26.074 [606] |______ 39.050 [4150] |________________________________________ 52.025 [3007] |_____________________________ 65.001 [789] |________ 77.976 [139] |_ 90.951 [10] | 103.927 [111] |_ 116.902 [133] |_ 129.877 [746] |_______ Latency distribution: 10% in 28.7386 secs 25% in 33.3900 secs 50% in 38.8671 secs 75% in 48.7103 secs 90% in 81.2245 secs 95% in 128.1317 secs 99% in 129.3828 secs Details (average, fastest, slowest): DNS+dialup: 32.5841 secs, 0.1238 secs, 129.8773 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.1551 secs resp wait: 14.4527 secs, 0.0760 secs, 129.1943 secs resp read: 0.0094 secs, 0.0001 secs, 1.1798 secs Status code distribution: [200] 8924 responses [503] 1032 responses Error distribution: [23] Get "https://www.electrobsd.org/": context deadline exceeded [21] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 523.1596 secs Slowest: 117.2104 secs Fastest: 1.1062 secs Average: 47.6696 secs Requests/sec: 19.1146 Total data: 316575 bytes Size/request: 32 bytes Response time histogram: 1.106 [1] | 12.717 [153] |_ 24.327 [296] |___ 35.937 [159] |_ 47.548 [4488] |________________________________________ 59.158 [3844] |__________________________________ 70.769 [817] |_______ 82.379 [60] |_ 93.990 [8] | 105.600 [3] | 117.210 [2] | Latency distribution: 10% in 39.0949 secs 25% in 42.7823 secs 50% in 47.1008 secs 75% in 55.3380 secs 90% in 58.9738 secs 95% in 60.5852 secs 99% in 68.4617 secs Details (average, fastest, slowest): DNS+dialup: 43.6173 secs, 1.1062 secs, 117.2104 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.0275 secs resp wait: 4.0496 secs, 0.1013 secs, 41.0136 secs resp read: 0.0024 secs, 0.0001 secs, 0.7430 secs Status code distribution: [200] 9786 responses [503] 45 responses Error distribution: [88] Get "https://www.electrobsd.org/": context deadline exceeded [81] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 347.0963 secs Slowest: 92.0937 secs Fastest: 0.0660 secs Average: 28.6340 secs Requests/sec: 28.8105 Total data: 513555 bytes Size/request: 51 bytes Response time histogram: 0.066 [1] | 9.269 [379] |__ 18.472 [92] |_ 27.674 [6341] |________________________________________ 36.877 [1724] |___________ 46.080 [559] |____ 55.283 [209] |_ 64.485 [308] |__ 73.688 [261] |__ 82.891 [2] | 92.094 [7] | Latency distribution: 10% in 21.9224 secs 25% in 23.7595 secs 50% in 25.6125 secs 75% in 29.0265 secs 90% in 41.5099 secs 95% in 62.4491 secs 99% in 65.2665 secs Details (average, fastest, slowest): DNS+dialup: 23.0281 secs, 0.0660 secs, 92.0937 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0039 secs, 0.0000 secs, 0.4656 secs resp wait: 5.5940 secs, 0.0575 secs, 41.0118 secs resp read: 0.0079 secs, 0.0001 secs, 1.0702 secs Status code distribution: [200] 9810 responses [503] 73 responses Error distribution: [117] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -t 0 -disable-keepalive https://www.electrobsd.org/ Summary: Total: 1066.2614 secs Slowest: 432.5674 secs Fastest: 0.1520 secs Average: 72.9039 secs Requests/sec: 9.3786 Total data: 928620 bytes Size/request: 93 bytes Response time histogram: 0.152 [1] | 43.394 [1172] |_______ 86.635 [6598] |________________________________________ 129.877 [2104] |_____________ 173.118 [4] | 216.360 [0] | 259.601 [0] | 302.843 [0] | 346.084 [0] | 389.326 [29] | 432.567 [53] | Latency distribution: 10% in 40.3065 secs 25% in 58.8633 secs 50% in 75.8038 secs 75% in 85.3442 secs 90% in 92.2249 secs 95% in 97.8203 secs 99% in 112.2393 secs Details (average, fastest, slowest): DNS+dialup: 58.6916 secs, 0.1520 secs, 432.5674 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.4757 secs, 0.0000 secs, 10.2244 secs resp wait: 13.3211 secs, 0.0702 secs, 363.8382 secs resp read: 0.4153 secs, 0.0002 secs, 12.5637 secs Status code distribution: [200] 9829 responses [503] 132 responses Error distribution: [39] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 6:21 | 381M |
| LibreSSL 3.2.3_2 | 6:21 | 413M |
| LibreSSL devel 3.3.1 | 6:56 | 367M |
| OpenSSL 1.1.1j,1 | 7:21 | 561M |
| OpenSSL devel 3.0.0.a11 | 7:23 | 284M |
| wolfSSL 4.7.0_2 | 4:11 | 278M |
| mbedtls 2.16.9_6 | 8:05 | 315M |
The OpenSSL devel port has been updated to 3.0.0.a12 and privoxy-experimental-ports-openssl-devel 3.0.32.20210216 has been recompiled.
In this test https://ads.electrobsd/ is requested and blocked by Privoxy. Connections are reused. Concurrency is reduced to 500 as generating the block page requires more memory.
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/ Summary: Total: 56.5757 secs Slowest: 17.3687 secs Fastest: 0.0022 secs Average: 1.5680 secs Requests/sec: 176.7542 Total data: 92320000 bytes Size/request: 9232 bytes Response time histogram: 0.002 [1] | 1.739 [7401] |________________________________________ 3.475 [877] |_____ 5.212 [854] |_____ 6.949 [326] |__ 8.685 [286] |__ 10.422 [163] |_ 12.159 [63] | 13.895 [16] | 15.632 [6] | 17.369 [7] | Latency distribution: 10% in 0.0062 secs 25% in 0.0926 secs 50% in 0.5230 secs 75% in 1.9266 secs 90% in 4.8700 secs 95% in 7.3835 secs 99% in 10.2803 secs Details (average, fastest, slowest): DNS+dialup: 0.0112 secs, 0.0022 secs, 17.3687 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0001 secs, 0.0000 secs, 0.0232 secs resp wait: 1.2270 secs, 0.0016 secs, 16.2375 secs resp read: 0.0092 secs, 0.0002 secs, 5.7104 secs Status code distribution: [403] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/ Summary: Total: 32.2621 secs Slowest: 32.0170 secs Fastest: 0.0019 secs Average: 1.2910 secs Requests/sec: 309.9613 Total data: 92320000 bytes Size/request: 9232 bytes Response time histogram: 0.002 [1] | 3.203 [9622] |________________________________________ 6.405 [90] | 9.606 [154] |_ 12.808 [27] | 16.009 [0] | 19.211 [20] | 22.412 [67] | 25.614 [1] | 28.815 [0] | 32.017 [18] | Latency distribution: 10% in 0.0038 secs 25% in 0.5269 secs 50% in 0.9290 secs 75% in 1.2685 secs 90% in 1.9006 secs 95% in 2.3871 secs 99% in 18.1151 secs Details (average, fastest, slowest): DNS+dialup: 0.0065 secs, 0.0019 secs, 32.0170 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.0366 secs resp wait: 0.5585 secs, 0.0015 secs, 31.4776 secs resp read: 0.0067 secs, 0.0000 secs, 8.0812 secs Status code distribution: [403] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/ Summary: Total: 29.8085 secs Slowest: 29.5272 secs Fastest: 0.0020 secs Average: 1.1402 secs Requests/sec: 335.4750 Total data: 92320000 bytes Size/request: 9232 bytes Response time histogram: 0.002 [1] | 2.954 [9646] |________________________________________ 5.907 [27] | 8.860 [29] | 11.812 [17] | 14.765 [46] | 17.717 [48] | 20.670 [86] | 23.622 [80] | 26.575 [12] | 29.527 [8] | Latency distribution: 10% in 0.0023 secs 25% in 0.0051 secs 50% in 0.7164 secs 75% in 0.9514 secs 90% in 1.3964 secs 95% in 2.1237 secs 99% in 20.7004 secs Details (average, fastest, slowest): DNS+dialup: 0.0112 secs, 0.0020 secs, 29.5272 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.0451 secs resp wait: 0.5848 secs, 0.0015 secs, 28.7065 secs resp read: 0.0021 secs, 0.0000 secs, 1.3987 secs Status code distribution: [403] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/ Summary: Total: 43.1997 secs Slowest: 33.1239 secs Fastest: 0.0019 secs Average: 1.3374 secs Requests/sec: 231.4831 Total data: 92320000 bytes Size/request: 9232 bytes Response time histogram: 0.002 [1] | 3.314 [9340] |________________________________________ 6.626 [365] |__ 9.939 [127] |_ 13.251 [46] | 16.563 [44] | 19.875 [48] | 23.187 [1] | 26.499 [0] | 29.812 [3] | 33.124 [25] | Latency distribution: 10% in 0.0036 secs 25% in 0.3672 secs 50% in 0.7277 secs 75% in 1.2631 secs 90% in 2.7113 secs 95% in 3.8144 secs 99% in 16.0184 secs Details (average, fastest, slowest): DNS+dialup: 0.0108 secs, 0.0019 secs, 33.1239 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.1298 secs resp wait: 0.6927 secs, 0.0015 secs, 32.5261 secs resp read: 0.0086 secs, 0.0000 secs, 5.3432 secs Status code distribution: [403] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/ Summary: Total: 36.9314 secs Slowest: 36.2355 secs Fastest: 0.0019 secs Average: 1.5423 secs Requests/sec: 270.7725 Total data: 92320000 bytes Size/request: 9232 bytes Response time histogram: 0.002 [1] | 3.625 [9474] |________________________________________ 7.249 [180] |_ 10.872 [275] |_ 14.495 [0] | 18.119 [1] | 21.742 [0] | 25.365 [16] | 28.989 [0] | 32.612 [3] | 36.236 [50] | Latency distribution: 10% in 0.0135 secs 25% in 0.8419 secs 50% in 1.0043 secs 75% in 1.5145 secs 90% in 2.0303 secs 95% in 6.5174 secs 99% in 8.0359 secs Details (average, fastest, slowest): DNS+dialup: 0.0050 secs, 0.0019 secs, 36.2355 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.0205 secs resp wait: 0.4955 secs, 0.0015 secs, 35.6578 secs resp read: 0.0075 secs, 0.0000 secs, 2.2044 secs Status code distribution: [403] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/ Summary: Total: 34.4870 secs Slowest: 20.2775 secs Fastest: 0.0022 secs Average: 1.4665 secs Requests/sec: 289.9647 Total data: 92320000 bytes Size/request: 9232 bytes Response time histogram: 0.002 [1] | 2.030 [8535] |________________________________________ 4.057 [781] |____ 6.085 [315] |_ 8.112 [107] |_ 10.140 [115] |_ 12.167 [33] | 14.195 [8] | 16.222 [47] | 18.250 [55] | 20.278 [3] | Latency distribution: 10% in 0.1146 secs 25% in 0.6212 secs 50% in 0.9245 secs 75% in 1.2583 secs 90% in 3.2655 secs 95% in 4.7252 secs 99% in 14.7234 secs Details (average, fastest, slowest): DNS+dialup: 0.0335 secs, 0.0022 secs, 20.2775 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0001 secs, 0.0000 secs, 0.0173 secs resp wait: 0.6001 secs, 0.0015 secs, 19.3268 secs resp read: 0.0058 secs, 0.0002 secs, 2.8281 secs Status code distribution: [403] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/ Summary: Total: 59.4872 secs Slowest: 37.5913 secs Fastest: 0.0019 secs Average: 2.7007 secs Requests/sec: 168.1035 Total data: 92320000 bytes Size/request: 9232 bytes Response time histogram: 0.002 [1] | 3.761 [8436] |________________________________________ 7.520 [726] |___ 11.279 [283] |_ 15.038 [51] | 18.797 [28] | 22.556 [12] | 26.314 [60] | 30.073 [121] |_ 33.832 [264] |_ 37.591 [18] | Latency distribution: 10% in 0.0027 secs 25% in 0.0537 secs 50% in 0.3232 secs 75% in 2.1293 secs 90% in 6.4967 secs 95% in 17.1389 secs 99% in 31.3368 secs Details (average, fastest, slowest): DNS+dialup: 0.2968 secs, 0.0019 secs, 37.5913 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0146 secs, 0.0000 secs, 2.6626 secs resp wait: 1.1153 secs, 0.0015 secs, 12.9543 secs resp read: 0.0015 secs, 0.0000 secs, 1.3732 secs Status code distribution: [403] 10000 responses
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 1:00 | 569M |
| LibreSSL 3.2.3_2 | 0:51 | 644M |
| LibreSSL devel 3.3.1 | 0:44 | 670M |
| OpenSSL 1.1.1j,1 | 1:04 | 728M |
| OpenSSL devel 3.0.0.a12 | 1:21 | 599M |
| wolfSSL 4.7.0_2 | 0:32 | 387M |
| mbedtls 2.16.9_6 | 0:45 | 111M |
Again the blocked https://ads.electrobsd/ is requested, this time without reusing connections.
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 506.8149 secs Slowest: 74.6107 secs Fastest: 1.1547 secs Average: 24.9626 secs Requests/sec: 19.7311 Total data: 92320000 bytes Size/request: 9232 bytes Response time histogram: 1.155 [1] | 8.500 [73] |_ 15.846 [171] |_ 23.192 [3563] |___________________________ 30.537 [5222] |________________________________________ 37.883 [351] |___ 45.228 [348] |___ 52.574 [246] |__ 59.920 [10] | 67.265 [0] | 74.611 [15] | Latency distribution: 10% in 20.3134 secs 25% in 21.5414 secs 50% in 24.1415 secs 75% in 26.0044 secs 90% in 30.3611 secs 95% in 39.9107 secs 99% in 49.1163 secs Details (average, fastest, slowest): DNS+dialup: 20.1254 secs, 1.1547 secs, 74.6107 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.4114 secs, 0.0000 secs, 2.9489 secs resp wait: 4.0431 secs, 0.0037 secs, 24.9994 secs resp read: 0.3802 secs, 0.0001 secs, 3.8255 secs Status code distribution: [403] 10000 responses
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 182.8982 secs Slowest: 58.4864 secs Fastest: 0.0087 secs Average: 5.0819 secs Requests/sec: 54.6752 Total data: 92052272 bytes Size/request: 9232 bytes Response time histogram: 0.009 [1] | 5.856 [8742] |________________________________________ 11.704 [1222] |______ 17.552 [0] | 23.400 [0] | 29.248 [2] | 35.095 [0] | 40.943 [2] | 46.791 [0] | 52.639 [0] | 58.486 [2] | Latency distribution: 10% in 3.9290 secs 25% in 5.0435 secs 50% in 5.2697 secs 75% in 5.5730 secs 90% in 5.9476 secs 95% in 6.2688 secs 99% in 7.3882 secs Details (average, fastest, slowest): DNS+dialup: 4.9526 secs, 0.0087 secs, 58.4864 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0011 secs, 0.0000 secs, 0.3014 secs resp wait: 0.1250 secs, 0.0017 secs, 2.3706 secs resp read: 0.0032 secs, 0.0001 secs, 0.8135 secs Status code distribution: [403] 9971 responses Error distribution: [29] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 183.2557 secs Slowest: 63.8003 secs Fastest: 0.0088 secs Average: 5.2874 secs Requests/sec: 54.5686 Total data: 92190752 bytes Size/request: 9232 bytes Response time histogram: 0.009 [1] | 6.388 [8788] |________________________________________ 12.767 [1192] |_____ 19.146 [1] | 25.525 [0] | 31.905 [0] | 38.284 [0] | 44.663 [3] | 51.042 [0] | 57.421 [0] | 63.800 [1] | Latency distribution: 10% in 3.6121 secs 25% in 4.9986 secs 50% in 5.4536 secs 75% in 5.7698 secs 90% in 6.5397 secs 95% in 7.2812 secs 99% in 8.4554 secs Details (average, fastest, slowest): DNS+dialup: 4.9393 secs, 0.0088 secs, 63.8003 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0009 secs, 0.0000 secs, 0.0832 secs resp wait: 0.3439 secs, 0.0017 secs, 3.0059 secs resp read: 0.0032 secs, 0.0001 secs, 0.6683 secs Status code distribution: [403] 9986 responses Error distribution: [14] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 160.6947 secs Slowest: 58.4727 secs Fastest: 0.0092 secs Average: 5.3160 secs Requests/sec: 62.2298 Total data: 92190752 bytes Size/request: 9232 bytes Response time histogram: 0.009 [1] | 5.856 [7818] |________________________________________ 11.702 [2156] |___________ 17.548 [0] | 23.395 [3] | 29.241 [1] | 35.087 [0] | 40.934 [4] | 46.780 [0] | 52.626 [0] | 58.473 [3] | Latency distribution: 10% in 3.8861 secs 25% in 5.0280 secs 50% in 5.4429 secs 75% in 5.7679 secs 90% in 6.7195 secs 95% in 7.3885 secs 99% in 9.1674 secs Details (average, fastest, slowest): DNS+dialup: 4.8543 secs, 0.0092 secs, 58.4727 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0012 secs, 0.0000 secs, 0.1361 secs resp wait: 0.4562 secs, 0.0020 secs, 4.1427 secs resp read: 0.0041 secs, 0.0001 secs, 1.0762 secs Status code distribution: [403] 9986 responses Error distribution: [14] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 158.0386 secs Slowest: 41.6518 secs Fastest: 0.0137 secs Average: 7.6646 secs Requests/sec: 63.2757 Total data: 92301536 bytes Size/request: 9232 bytes Response time histogram: 0.014 [1] | 4.178 [445] |___ 8.341 [6622] |________________________________________ 12.505 [2807] |_________________ 16.669 [122] |_ 20.833 [0] | 24.997 [0] | 29.160 [0] | 33.324 [0] | 37.488 [0] | 41.652 [1] | Latency distribution: 10% in 5.7120 secs 25% in 6.7147 secs 50% in 7.4759 secs 75% in 8.5631 secs 90% in 10.2541 secs 95% in 11.2007 secs 99% in 13.5611 secs Details (average, fastest, slowest): DNS+dialup: 6.6690 secs, 0.0137 secs, 41.6518 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0010 secs, 0.0000 secs, 0.1767 secs resp wait: 0.9755 secs, 0.0022 secs, 6.5447 secs resp read: 0.0190 secs, 0.0001 secs, 3.5642 secs Status code distribution: [403] 9998 responses Error distribution: [2] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 228.6437 secs Slowest: 58.4782 secs Fastest: 0.0079 secs Average: 4.2100 secs Requests/sec: 43.7362 Total data: 90076624 bytes Size/request: 9232 bytes Response time histogram: 0.008 [1] | 5.855 [8476] |________________________________________ 11.702 [1270] |______ 17.549 [1] | 23.396 [0] | 29.243 [0] | 35.090 [7] | 40.937 [0] | 46.784 [0] | 52.631 [0] | 58.478 [2] | Latency distribution: 10% in 1.0798 secs 25% in 3.2238 secs 50% in 4.6062 secs 75% in 5.3328 secs 90% in 6.0933 secs 95% in 6.6897 secs 99% in 7.3967 secs Details (average, fastest, slowest): DNS+dialup: 3.6402 secs, 0.0079 secs, 58.4782 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0008 secs, 0.0000 secs, 0.1308 secs resp wait: 0.5608 secs, 0.0017 secs, 3.4118 secs resp read: 0.0082 secs, 0.0002 secs, 1.2066 secs Status code distribution: [403] 9757 responses Error distribution: [243] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 512.6566 secs Slowest: 62.0966 secs Fastest: 0.1080 secs Average: 25.0630 secs Requests/sec: 19.5062 Total data: 92301536 bytes Size/request: 9232 bytes Response time histogram: 0.108 [1] | 6.307 [24] | 12.506 [109] |_ 18.705 [284] |__ 24.903 [5274] |________________________________________ 31.102 [3479] |__________________________ 37.301 [289] |__ 43.500 [313] |__ 49.699 [133] |_ 55.898 [90] |_ 62.097 [2] | Latency distribution: 10% in 20.9160 secs 25% in 21.7648 secs 50% in 24.4792 secs 75% in 25.8387 secs 90% in 29.4425 secs 95% in 38.0777 secs 99% in 49.4553 secs Details (average, fastest, slowest): DNS+dialup: 20.2104 secs, 0.1080 secs, 62.0966 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.4061 secs, 0.0000 secs, 2.5067 secs resp wait: 4.0624 secs, 0.0019 secs, 31.0060 secs resp read: 0.3785 secs, 0.0002 secs, 5.1205 secs Status code distribution: [403] 9998 responses Error distribution: [2] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t (maybe not?) | 5:22 | 187M |
| LibreSSL 3.2.3_2 | 1:38 | 230M |
| LibreSSL devel 3.3.1 | 1:41 | 264M |
| OpenSSL 1.1.1j,1 | 1:41 | 191M |
| OpenSSL devel 3.0.0.a12 | 2:21 | 277M |
| wolfSSL 4.7.0_2 | 1:23 | 292M |
| mbedtls 2.16.9_6 | 4:42 | 172M |
The results for OpenSSL 1.0.2t
are clearly suspect. I wonder if I
they can be explained by changed background noise on the host of if I accidentally
continued to test with mbedtls 2.16.9_6 installed from a previous run.
The wolfssl port has been updated to include --enable-intelasm and then updated again to remove the configure flag as the resulting binaries don't work on the test system. privoxy-experimental-wolfssl 3.0.32.20210216_1 has been recompiled. Twice.
As an experiment I configured Privoxy to enable the HTTP accept filter to see if it makes a difference for CONNECT requests. The option can only impact new connections so I didn't runt the tests again with keep-alive enabled.
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 196.7352 secs Slowest: 67.1195 secs Fastest: 0.0106 secs Average: 8.3404 secs Requests/sec: 50.8297 Total data: 92264608 bytes Size/request: 9232 bytes Response time histogram: 0.011 [1] | 6.721 [1330] |______ 13.432 [8546] |________________________________________ 20.143 [115] |_ 26.854 [0] | 33.565 [0] | 40.276 [0] | 46.987 [0] | 53.698 [0] | 60.409 [0] | 67.120 [2] | Latency distribution: 10% in 6.2528 secs 25% in 7.3807 secs 50% in 8.3670 secs 75% in 9.3534 secs 90% in 10.7378 secs 95% in 11.5283 secs 99% in 13.5000 secs Details (average, fastest, slowest): DNS+dialup: 7.5155 secs, 0.0106 secs, 67.1195 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0021 secs, 0.0000 secs, 0.5894 secs resp wait: 0.8038 secs, 0.0019 secs, 4.2702 secs resp read: 0.0187 secs, 0.0002 secs, 1.8232 secs Status code distribution: [403] 9994 responses Error distribution: [6] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 219.6350 secs Slowest: 16.0899 secs Fastest: 0.0106 secs Average: 7.7134 secs Requests/sec: 45.5301 Total data: 92209216 bytes Size/request: 9232 bytes Response time histogram: 0.011 [1] | 1.618 [114] |_ 3.226 [282] |___ 4.834 [208] |__ 6.442 [694] |______ 8.050 [4309] |________________________________________ 9.658 [3419] |________________________________ 11.266 [844] |________ 12.874 [43] | 14.482 [50] | 16.090 [24] | Latency distribution: 10% in 6.1966 secs 25% in 7.0436 secs 50% in 7.8535 secs 75% in 8.5956 secs 90% in 9.6273 secs 95% in 10.1041 secs 99% in 11.4976 secs Details (average, fastest, slowest): DNS+dialup: 7.1040 secs, 0.0106 secs, 16.0899 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0018 secs, 0.0000 secs, 0.2055 secs resp wait: 0.5928 secs, 0.0019 secs, 7.1156 secs resp read: 0.0146 secs, 0.0002 secs, 3.8758 secs Status code distribution: [403] 9988 responses Error distribution: [12] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 229.1791 secs Slowest: 66.5862 secs Fastest: 0.0102 secs Average: 7.4500 secs Requests/sec: 43.6340 Total data: 91895328 bytes Size/request: 9232 bytes Response time histogram: 0.010 [1] | 6.668 [2700] |_______________ 13.325 [7116] |________________________________________ 19.983 [51] | 26.641 [11] | 33.298 [7] | 39.956 [9] | 46.613 [11] | 53.271 [2] | 59.929 [29] | 66.586 [17] | Latency distribution: 10% in 3.6489 secs 25% in 6.5623 secs 50% in 7.5110 secs 75% in 8.3671 secs 90% in 9.3577 secs 95% in 10.1053 secs 99% in 19.6965 secs Details (average, fastest, slowest): DNS+dialup: 6.9290 secs, 0.0102 secs, 66.5862 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0019 secs, 0.0000 secs, 0.4935 secs resp wait: 0.5125 secs, 0.0019 secs, 7.8039 secs resp read: 0.0064 secs, 0.0001 secs, 3.2971 secs Status code distribution: [403] 9954 responses Error distribution: [46] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 175.7938 secs Slowest: 68.1560 secs Fastest: 0.0358 secs Average: 8.3704 secs Requests/sec: 56.8848 Total data: 92227680 bytes Size/request: 9232 bytes Response time histogram: 0.036 [1] | 6.848 [1356] |______ 13.660 [8580] |________________________________________ 20.472 [44] | 27.284 [1] | 34.096 [0] | 40.908 [0] | 47.720 [3] | 54.532 [0] | 61.344 [0] | 68.156 [5] | Latency distribution: 10% in 6.3961 secs 25% in 7.5922 secs 50% in 8.4238 secs 75% in 9.4106 secs 90% in 10.5465 secs 95% in 11.0188 secs 99% in 12.6088 secs Details (average, fastest, slowest): DNS+dialup: 7.6406 secs, 0.0358 secs, 68.1560 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0055 secs, 0.0000 secs, 1.3088 secs resp wait: 0.7137 secs, 0.0022 secs, 7.5887 secs resp read: 0.0104 secs, 0.0001 secs, 4.9130 secs Status code distribution: [403] 9990 responses Error distribution: [10] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 237.6310 secs Slowest: 58.4496 secs Fastest: 0.0145 secs Average: 11.3357 secs Requests/sec: 42.0820 Total data: 92273840 bytes Size/request: 9232 bytes Response time histogram: 0.014 [1] | 5.858 [335] |__ 11.702 [5422] |________________________________________ 17.545 [4122] |______________________________ 23.389 [111] |_ 29.232 [3] | 35.076 [0] | 40.919 [0] | 46.763 [0] | 52.606 [0] | 58.450 [1] | Latency distribution: 10% in 8.7541 secs 25% in 9.9750 secs 50% in 11.2747 secs 75% in 12.7297 secs 90% in 14.4656 secs 95% in 15.5462 secs 99% in 17.7227 secs Details (average, fastest, slowest): DNS+dialup: 10.1225 secs, 0.0145 secs, 58.4496 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0024 secs, 0.0000 secs, 0.2212 secs resp wait: 1.1980 secs, 0.0021 secs, 7.3346 secs resp read: 0.0126 secs, 0.0001 secs, 3.0961 secs Status code distribution: [403] 9995 responses Error distribution: [5] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 234.1005 secs Slowest: 12.5116 secs Fastest: 0.0100 secs Average: 7.4811 secs Requests/sec: 42.7167 Total data: 92190752 bytes Size/request: 9232 bytes Response time histogram: 0.010 [1] | 1.260 [139] |__ 2.510 [133] |__ 3.760 [114] |_ 5.011 [281] |___ 6.261 [849] |__________ 7.511 [3262] |________________________________________ 8.761 [3283] |________________________________________ 10.011 [1405] |_________________ 11.261 [495] |______ 12.512 [24] | Latency distribution: 10% in 5.7744 secs 25% in 6.7763 secs 50% in 7.5677 secs 75% in 8.5206 secs 90% in 9.4748 secs 95% in 10.0378 secs 99% in 10.8210 secs Details (average, fastest, slowest): DNS+dialup: 6.4370 secs, 0.0100 secs, 12.5116 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0040 secs, 0.0000 secs, 0.5011 secs resp wait: 1.0207 secs, 0.0020 secs, 4.1652 secs resp read: 0.0192 secs, 0.0003 secs, 0.9506 secs Status code distribution: [403] 9986 responses Error distribution: [14] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 681.7736 secs Slowest: 90.7298 secs Fastest: 0.2496 secs Average: 33.5403 secs Requests/sec: 14.6676 Total data: 92310768 bytes Size/request: 9232 bytes Response time histogram: 0.250 [1] | 9.298 [63] | 18.346 [151] |_ 27.394 [1104] |_______ 36.442 [6191] |________________________________________ 45.490 [1965] |_____________ 54.538 [101] |_ 63.586 [327] |__ 72.634 [93] |_ 81.682 [2] | 90.730 [1] | Latency distribution: 10% in 26.1072 secs 25% in 29.3204 secs 50% in 33.0000 secs 75% in 36.4320 secs 90% in 40.0730 secs 95% in 47.3563 secs 99% in 62.9468 secs Details (average, fastest, slowest): DNS+dialup: 27.9099 secs, 0.2496 secs, 90.7298 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.5864 secs, 0.0000 secs, 17.8840 secs resp wait: 4.4436 secs, 0.0028 secs, 35.6120 secs resp read: 0.5844 secs, 0.0001 secs, 9.0368 secs Status code distribution: [403] 9999 responses Error distribution: [1] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 2:12 | 316M |
| LibreSSL 3.2.3_2 | 3:44 | 500M |
| LibreSSL devel 3.3.1 | 2:17 | 279M |
| OpenSSL 1.1.1j,1 | 2:17 | 204M |
| OpenSSL devel 3.0.0.a12 | 3:09 | 259M |
| wolfSSL 4.7.0_4 | 2:05 | 263M |
| mbedtls 2.16.9_6 | 5:27 | 137M |
The results for OpenSSL 1.0.2t look reasonable again.
I repeated the test with the same Privoxy configuration but shutdown a bhyve machine first that was started on 2020-02-19.
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 229.1192 secs Slowest: 14.7169 secs Fastest: 0.0103 secs Average: 7.2841 secs Requests/sec: 43.6454 Total data: 92107664 bytes Size/request: 9232 bytes Response time histogram: 0.010 [1] | 1.481 [204] |__ 2.952 [117] |_ 4.422 [246] |___ 5.893 [883] |__________ 7.364 [3435] |________________________________________ 8.834 [3440] |________________________________________ 10.305 [1224] |______________ 11.776 [356] |____ 13.246 [55] |_ 14.717 [16] | Latency distribution: 10% in 5.1309 secs 25% in 6.5465 secs 50% in 7.3978 secs 75% in 8.2579 secs 90% in 9.3621 secs 95% in 10.1418 secs 99% in 11.3833 secs Details (average, fastest, slowest): DNS+dialup: 6.5283 secs, 0.0103 secs, 14.7169 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0022 secs, 0.0000 secs, 0.1999 secs resp wait: 0.7431 secs, 0.0019 secs, 6.5206 secs resp read: 0.0102 secs, 0.0002 secs, 0.7722 secs Status code distribution: [403] 9977 responses Error distribution: [23] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 201.5610 secs Slowest: 67.3539 secs Fastest: 0.0105 secs Average: 8.5295 secs Requests/sec: 49.6128 Total data: 92209216 bytes Size/request: 9232 bytes Response time histogram: 0.010 [1] | 6.745 [1674] |_________ 13.479 [7798] |________________________________________ 20.214 [174] |_ 26.948 [338] |__ 33.682 [1] | 40.417 [0] | 47.151 [0] | 53.885 [0] | 60.620 [0] | 67.354 [2] | Latency distribution: 10% in 5.8359 secs 25% in 7.2752 secs 50% in 8.1401 secs 75% in 9.0929 secs 90% in 10.6214 secs 95% in 14.3723 secs 99% in 22.0539 secs Details (average, fastest, slowest): DNS+dialup: 7.6579 secs, 0.0105 secs, 67.3539 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0061 secs, 0.0000 secs, 0.4268 secs resp wait: 0.8436 secs, 0.0020 secs, 11.2957 secs resp read: 0.0217 secs, 0.0001 secs, 6.1869 secs Status code distribution: [403] 9988 responses Error distribution: [12] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 224.9795 secs Slowest: 59.2200 secs Fastest: 0.0105 secs Average: 6.1113 secs Requests/sec: 44.4485 Total data: 90547456 bytes Size/request: 9232 bytes Response time histogram: 0.011 [1] | 5.931 [3293] |____________________ 11.852 [6458] |________________________________________ 17.773 [21] | 23.694 [0] | 29.615 [22] | 35.536 [4] | 41.457 [5] | 47.378 [0] | 53.299 [0] | 59.220 [4] | Latency distribution: 10% in 0.8811 secs 25% in 5.5868 secs 50% in 6.5774 secs 75% in 7.7550 secs 90% in 8.7496 secs 95% in 9.2552 secs 99% in 11.1063 secs Details (average, fastest, slowest): DNS+dialup: 5.6280 secs, 0.0105 secs, 59.2200 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0030 secs, 0.0000 secs, 0.3329 secs resp wait: 0.4721 secs, 0.0020 secs, 6.7668 secs resp read: 0.0080 secs, 0.0001 secs, 0.6305 secs Status code distribution: [403] 9808 responses Error distribution: [192] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 206.1265 secs Slowest: 67.3193 secs Fastest: 0.0117 secs Average: 8.4849 secs Requests/sec: 48.5139 Total data: 92126128 bytes Size/request: 9232 bytes Response time histogram: 0.012 [1] | 6.742 [1077] |_____ 13.473 [8823] |________________________________________ 20.204 [73] | 26.935 [1] | 33.666 [0] | 40.396 [0] | 47.127 [1] | 53.858 [0] | 60.589 [0] | 67.319 [3] | Latency distribution: 10% in 6.6289 secs 25% in 7.7299 secs 50% in 8.6040 secs 75% in 9.5682 secs 90% in 10.3730 secs 95% in 11.1952 secs 99% in 13.1969 secs Details (average, fastest, slowest): DNS+dialup: 7.7028 secs, 0.0117 secs, 67.3193 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0043 secs, 0.0000 secs, 0.5182 secs resp wait: 0.7650 secs, 0.0020 secs, 8.2382 secs resp read: 0.0126 secs, 0.0001 secs, 0.8065 secs Status code distribution: [403] 9979 responses Error distribution: [21] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 277.7110 secs Slowest: 73.4518 secs Fastest: 0.0144 secs Average: 9.9900 secs Requests/sec: 36.0087 Total data: 91138304 bytes Size/request: 9232 bytes Response time histogram: 0.014 [1] | 7.358 [1778] |_________ 14.702 [7571] |________________________________________ 22.046 [479] |___ 29.389 [5] | 36.733 [19] | 44.077 [10] | 51.421 [1] | 58.764 [0] | 66.108 [0] | 73.452 [8] | Latency distribution: 10% in 4.5968 secs 25% in 8.2476 secs 50% in 10.1133 secs 75% in 12.2018 secs 90% in 13.9013 secs 95% in 14.7525 secs 99% in 17.2616 secs Details (average, fastest, slowest): DNS+dialup: 8.9541 secs, 0.0144 secs, 73.4518 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0024 secs, 0.0000 secs, 0.2722 secs resp wait: 1.0195 secs, 0.0021 secs, 8.3752 secs resp read: 0.0137 secs, 0.0001 secs, 0.8761 secs Status code distribution: [403] 9872 responses Error distribution: [128] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 243.3433 secs Slowest: 66.2667 secs Fastest: 0.0092 secs Average: 7.1743 secs Requests/sec: 41.0942 Total data: 91941488 bytes Size/request: 9232 bytes Response time histogram: 0.009 [1] | 6.635 [3403] |_____________________ 13.261 [6552] |________________________________________ 19.886 [0] | 26.512 [0] | 33.138 [0] | 39.764 [0] | 46.389 [1] | 53.015 [0] | 59.641 [1] | 66.267 [1] | Latency distribution: 10% in 5.1104 secs 25% in 6.2547 secs 50% in 7.3163 secs 75% in 8.3779 secs 90% in 9.4554 secs 95% in 10.1959 secs 99% in 11.5535 secs Details (average, fastest, slowest): DNS+dialup: 6.1148 secs, 0.0092 secs, 66.2667 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0042 secs, 0.0000 secs, 0.5503 secs resp wait: 1.0366 secs, 0.0021 secs, 4.0304 secs resp read: 0.0186 secs, 0.0003 secs, 1.2941 secs Status code distribution: [403] 9959 responses Error distribution: [41] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 -disable-keepalive https://ads.electrobsd.org/ Summary: Total: 651.4387 secs Slowest: 71.7645 secs Fastest: 0.1185 secs Average: 31.9575 secs Requests/sec: 15.3506 Total data: 92273840 bytes Size/request: 9232 bytes Response time histogram: 0.118 [1] | 7.283 [42] | 14.448 [64] |_ 21.612 [356] |___ 28.777 [2539] |_____________________ 35.941 [4943] |________________________________________ 43.106 [1524] |____________ 50.271 [255] |__ 57.435 [175] |_ 64.600 [76] |_ 71.764 [20] | Latency distribution: 10% in 24.8700 secs 25% in 28.1502 secs 50% in 31.5823 secs 75% in 35.0529 secs 90% in 39.1698 secs 95% in 43.6228 secs 99% in 57.2514 secs Details (average, fastest, slowest): DNS+dialup: 26.3336 secs, 0.1185 secs, 71.7645 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.5315 secs, 0.0000 secs, 5.1526 secs resp wait: 4.4777 secs, 0.0024 secs, 28.6529 secs resp read: 0.5911 secs, 0.0001 secs, 6.7086 secs Status code distribution: [403] 9995 responses Error distribution: [5] Get "https://ads.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 2:04 | 293M |
| LibreSSL 3.2.3_2 | 2:18 | 299M |
| LibreSSL devel 3.3.1 | 2:21 | 246M |
| OpenSSL 1.1.1j,1 | 2:21 | 178M |
| OpenSSL devel 3.0.0.a12 | 3:09 | 177M |
| wolfSSL 4.7.0_4 | 2:04 | 272M |
| mbedtls 2.16.9_6 | 5:44 | 170M |
It's not obvious to me why both hey and ab-proxy claim that some responses are supposedly really slow and I'm not sure if it's Privoxy's fault.
Also this test (with OpenSSL 1.0.2t):
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 500 -t 0 https://ads.electrobsd.org/ Summary: Total: 60.8107 secs Slowest: 38.0051 secs Fastest: 0.0026 secs Average: 2.9465 secs Requests/sec: 164.4446 Total data: 92320000 bytes Size/request: 9232 bytes Response time histogram: 0.003 [1] | 3.803 [8490] |________________________________________ 7.603 [1497] |_______ 11.403 [7] | 15.204 [1] | 19.004 [0] | 22.804 [3] | 26.604 [0] | 30.405 [0] | 34.205 [0] | 38.005 [1] | Latency distribution: 10% in 2.0306 secs 25% in 2.3515 secs 50% in 2.7079 secs 75% in 3.1667 secs 90% in 4.2272 secs 95% in 5.3383 secs 99% in 5.7358 secs Details (average, fastest, slowest): DNS+dialup: 0.0011 secs, 0.0026 secs, 38.0051 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0001 secs, 0.0000 secs, 0.0337 secs resp wait: 0.1759 secs, 0.0017 secs, 35.1991 secs resp read: 0.0037 secs, 0.0002 secs, 2.3429 secs Status code distribution: [403] 10000 responses
resulted in the following request distribution which does not seem ideal:
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --statistics /var/log/privoxy/privoxy.log
Client requests total: 10000
Crunches: 10000 (100.00%)
Blocks: 10000 (100.00%)
Fast redirections: 0 (0.00%)
Connection timeouts: 0 (0.00%)
Connection failures: 0 (0.00%)
Outgoing requests: 0 (0.00%)
Server keep-alive offers: 0 (0.00%)
New outgoing connections: 0 (0.00%)
Reused connections: 0 (0.00%; server offers accepted: 0.00%)
Empty responses: 0 (0.00%)
Empty responses on new connections: 0 (0.00%)
Empty responses on reused connections: 0 (0.00%)
Client connections: 3034
Bytes of content transfered to the client: 92320000
Client requests per connection distribution:
2995: 1
2: 330
1: 20
1: 381
1: 229
1: 255
1: 327
1: 277
1: 219
1: 201
Enable --show-complete-request-distribution to get less common numbers as well.
Improperly accounted requests: ~2992
Method distribution:
10000 : GET
Client HTTP versions:
10000 : HTTP/1.1
HTTP status codes:
10000 : 403
URL statistics are disabled. Increase --url-statistics-threshold to enable them.
Passed request statistics are disabled. Increase --passed-request-statistics-threshold to enable them.
Host statistics are disabled. Increase --host-statistics-threshold to enable them.
Apparently hey used a single connection to send 2995 of the 10000 requests instead of reusing the connections evenly. This could indicate that hey is preferring some of the connections when reading incoming requests which would skew the results.
I briefly looked at the hey code but I'm not used to debugging go programs and therefore didn't investigate this further.
I should probably try another benchmarking tool written in another language to see how the results differ.
There's work in progress to release Privoxy 3.0.32 so I've updated the ports to include current commits from git a73211c9fa. The Privoxy code hasn't changed much. I've changed the Privoxy configuration to include regression-tests.action.
The test times Privoxy-Regression-Test loading and processing various CGI resources through https. Privoxy does not make outgoing connnections.
Privoxy-Regression-Test runs curl commands one at a time, the concurrency should therefore be one and connections are not reused.
The test is executed on the system running Privoxy. Two test failures are currently
documented to be expected to occur when using --privoxy-cgi-prefix https://p.p/.
[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/ 2021-02-25 18:09:10: Asking Privoxy for the number of action files available ... 2021-02-25 18:09:10: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32. 2021-02-25 18:09:10: Executing regression tests ... 2021-02-25 18:09:32: Ooops. Expected removal but: 'Connection: close' is still there. 2021-02-25 18:09:32: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled' 2021-02-25 18:09:32: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE' 2021-02-25 18:09:32: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host' 2021-02-25 18:09:36: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures. real 0m26.307s user 0m13.953s sys 0m5.295s
[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/ 2021-02-25 18:13:07: Asking Privoxy for the number of action files available ... 2021-02-25 18:13:07: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32. 2021-02-25 18:13:07: Executing regression tests ... 2021-02-25 18:13:32: Ooops. Expected removal but: 'Connection: close' is still there. 2021-02-25 18:13:32: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled' 2021-02-25 18:13:32: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE' 2021-02-25 18:13:32: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host' 2021-02-25 18:13:36: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures. real 0m29.660s user 0m13.915s sys 0m5.379s
[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/ 2021-02-25 18:13:07: Asking Privoxy for the number of action files available ... 2021-02-25 18:13:07: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32. 2021-02-25 18:13:07: Executing regression tests ... 2021-02-25 18:13:32: Ooops. Expected removal but: 'Connection: close' is still there. 2021-02-25 18:13:32: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled' 2021-02-25 18:13:32: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE' 2021-02-25 18:13:32: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host' 2021-02-25 18:13:36: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures. real 0m29.660s user 0m13.915s sys 0m5.379s
[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/ 2021-02-25 18:15:08: Asking Privoxy for the number of action files available ... 2021-02-25 18:15:09: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32. 2021-02-25 18:15:09: Executing regression tests ... 2021-02-25 18:15:38: Ooops. Expected removal but: 'Connection: close' is still there. 2021-02-25 18:15:38: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled' 2021-02-25 18:15:38: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE' 2021-02-25 18:15:38: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host' 2021-02-25 18:15:42: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures. real 0m33.438s user 0m15.856s sys 0m5.449s
[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/ 2021-02-25 18:38:25: Asking Privoxy for the number of action files available ... 2021-02-25 18:38:25: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32. 2021-02-25 18:38:25: Executing regression tests ... 2021-02-25 18:38:52: Ooops. Expected removal but: 'Connection: close' is still there. 2021-02-25 18:38:52: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled' 2021-02-25 18:38:52: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE' 2021-02-25 18:38:52: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host' 2021-02-25 18:38:56: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures. real 0m31.408s user 0m15.492s sys 0m5.897s
[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/ 2021-02-25 18:18:48: Asking Privoxy for the number of action files available ... 2021-02-25 18:18:48: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32. 2021-02-25 18:18:48: Executing regression tests ... 2021-02-25 18:19:11: Ooops. Expected removal but: 'Connection: close' is still there. 2021-02-25 18:19:11: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled' 2021-02-25 18:19:11: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE' 2021-02-25 18:19:11: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host' 2021-02-25 18:19:14: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures. real 0m26.378s user 0m14.430s sys 0m5.724s
[fk@privoxy-vm ~]$ time privoxy-regression-test.pl --privoxy-cgi-prefix https://p.p/ 2021-02-25 18:20:17: Asking Privoxy for the number of action files available ... 2021-02-25 18:20:17: Gathering regression tests from 4 action file(s) delivered by Privoxy 3.0.32. 2021-02-25 18:20:17: Executing regression tests ... 2021-02-25 18:20:58: Ooops. Expected removal but: 'Connection: close' is still there. 2021-02-25 18:20:58: Failure for test 574. Header 'Connection: close' and tag 'connection-sharing enabled' 2021-02-25 18:20:58: Ooops. Got: 'Host: p.p' while expecting: 'NO CHANGE' 2021-02-25 18:20:58: Failure for test 577. Header 'Host: whatever.example.org' and tag 'Host header other than the target host' 2021-02-25 18:21:04: Executed 700 regression tests. Skipped 91. 698 successes, 2 failures. real 0m47.105s user 0m23.516s sys 0m5.845s
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 0:06 | 61804K |
| LibreSSL 3.2.3_2 | 0:09 | 62384K |
| LibreSSL devel 3.3.1 | 0:10 | 63388K |
| OpenSSL 1.1.1j,1 | 0:06 | 66108K |
| OpenSSL devel 3.0.0.a12 | 0:09 | 64376K |
| wolfSSL 4.7.0_4 | 0:05 | 63696K |
| mbedtls 2.16.9_6 | 0:16 | 33848K |
The Privoxy ports have been updated. One experimental change worth mentioning is that the memory used to temporarily store the certificates used by servers is allocated dynamically which could reduce the peak memory usage for tests where Privoxy does not answer the request itself.
Privoxy's wolfSSL version is expected to profit less from this because it already contains another optimization and only stores the certificates if the certificate chain could not be validated (the only case where the certificates are needed so the user can inspect them).
The OpenSSL devel port has been updated to 3.0.0.a13. The mbedTLS port has been updated to 2.16.10. The wolfSSL port contains a patch to disable ciphers using 3DES.
The ports on the benchmark VM have been upgraded as well: hey 0.1.3 has been replaced with 0.1.4 and go went from 1.14.7,1 to 1.15.6,1.
The binaries are dynamically linked so the sizes aren't significantly affected by the choice of the TLS library.
| TLS library | Privoxy binary size |
|---|---|
| OpenSSL 1.0.2t | 321K |
| LibreSSL 3.2.3_2 | 321K |
| LibreSSL devel 3.3.1 | 321K |
| OpenSSL 1.1.1j,1 | 321K |
| OpenSSL devel 3.0.0.a13 | 321K |
| wolfSSL 4.7.0_6 | 317K |
| mbedtls 2.16.10 | 317K |
As hey and go have been upgraded I decided to give them another try to see if the skewed results issue still exists. Spoiler alert: it does.
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/ Summary: Total: 483.1075 secs Slowest: 86.0884 secs Fastest: 7.6978 secs Average: 43.8206 secs Requests/sec: 20.6993 Total data: 204015 bytes Size/request: 21 bytes Response time histogram: 7.698 [1] | 15.537 [53] |_ 23.376 [38] | 31.215 [506] |_____ 39.054 [2296] |______________________ 46.893 [4093] |________________________________________ 54.732 [713] |_______ 62.571 [1020] |__________ 70.410 [452] |____ 78.249 [103] |_ 86.088 [15] | Latency distribution: 10% in 32.4097 secs 25% in 37.3129 secs 50% in 42.6137 secs 75% in 46.8448 secs 90% in 60.2343 secs 95% in 63.6705 secs 99% in 72.0302 secs Details (average, fastest, slowest): DNS+dialup: 38.6265 secs, 7.6978 secs, 86.0884 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0002 secs, 0.0000 secs, 0.0434 secs resp wait: 5.1896 secs, 0.1007 secs, 42.9234 secs resp read: 0.0043 secs, 0.0001 secs, 1.8764 secs Status code distribution: [200] 9261 responses [503] 29 responses Error distribution: [710] Get "https://www.electrobsd.org/": context deadline exceeded
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/ Summary: Total: 507.0580 secs Slowest: 113.1922 secs Fastest: 0.7074 secs Average: 45.6618 secs Requests/sec: 19.7216 Total data: 274365 bytes Size/request: 27 bytes Response time histogram: 0.707 [1] | 11.956 [160] |_ 23.204 [357] |___ 34.453 [352] |___ 45.701 [4510] |________________________________________ 56.950 [3242] |_____________________________ 68.198 [1173] |__________ 79.447 [127] |_ 90.695 [20] | 101.944 [9] | 113.192 [3] | Latency distribution: 10% in 34.9347 secs 25% in 39.9600 secs 50% in 44.6978 secs 75% in 54.6003 secs 90% in 57.8763 secs 95% in 60.9433 secs 99% in 73.1241 secs Details (average, fastest, slowest): DNS+dialup: 40.9759 secs, 0.7074 secs, 113.1922 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0005 secs, 0.0000 secs, 0.1712 secs resp wait: 4.6827 secs, 0.1726 secs, 44.5401 secs resp read: 0.0026 secs, 0.0001 secs, 1.4885 secs Status code distribution: [200] 9915 responses [503] 39 responses Error distribution: [25] Get "https://www.electrobsd.org/": context deadline exceeded [21] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/ Summary: Total: 732.1350 secs Slowest: 326.2141 secs Fastest: 0.5917 secs Average: 59.4638 secs Requests/sec: 13.6587 Total data: 2076981 bytes Size/request: 258 bytes Response time histogram: 0.592 [1] | 33.154 [572] |___ 65.716 [6573] |________________________________________ 98.278 [436] |___ 130.841 [1] | 163.403 [0] | 195.965 [1] | 228.527 [1] | 261.090 [1] | 293.652 [3] | 326.214 [435] |___ Latency distribution: 10% in 34.8167 secs 25% in 38.3919 secs 50% in 45.0731 secs 75% in 55.3488 secs 90% in 68.4707 secs 95% in 298.6087 secs 99% in 309.1964 secs Details (average, fastest, slowest): DNS+dialup: 41.8909 secs, 0.5917 secs, 326.2141 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0003 secs, 0.0000 secs, 0.0389 secs resp wait: 17.5563 secs, 0.1327 secs, 290.8596 secs resp read: 0.0162 secs, 0.0001 secs, 26.0727 secs Status code distribution: [200] 7733 responses [502] 52 responses [503] 239 responses Error distribution: [1963] Get "https://www.electrobsd.org/": context deadline exceeded [13] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/ Summary: Total: 490.4667 secs Slowest: 121.6341 secs Fastest: 0.4485 secs Average: 42.8682 secs Requests/sec: 20.3887 Total data: 316575 bytes Size/request: 34 bytes Response time histogram: 0.449 [1] | 12.567 [187] |__ 24.686 [195] |__ 36.804 [2174] |_______________________ 48.923 [3838] |________________________________________ 61.041 [2325] |________________________ 73.160 [429] |____ 85.278 [18] | 97.397 [13] | 109.516 [1] | 121.634 [3] | Latency distribution: 10% in 32.1865 secs 25% in 36.3391 secs 50% in 41.3034 secs 75% in 50.4415 secs 90% in 54.9276 secs 95% in 61.1024 secs 99% in 69.0942 secs Details (average, fastest, slowest): DNS+dialup: 37.7823 secs, 0.4485 secs, 121.6341 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0003 secs, 0.0000 secs, 0.0138 secs resp wait: 5.0817 secs, 0.1188 secs, 55.3515 secs resp read: 0.0039 secs, 0.0001 secs, 3.7078 secs Status code distribution: [200] 9139 responses [503] 45 responses Error distribution: [770] Get "https://www.electrobsd.org/": context deadline exceeded [46] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/ Summary: Total: 646.4045 secs Slowest: 224.0558 secs Fastest: 0.1815 secs Average: 58.4754 secs Requests/sec: 15.4702 Total data: 961599 bytes Size/request: 107 bytes Response time histogram: 0.181 [1] | 22.569 [524] |____ 44.956 [1446] |__________ 67.344 [5768] |________________________________________ 89.731 [356] |__ 112.119 [218] |__ 134.506 [166] |_ 156.893 [136] |_ 179.281 [111] |_ 201.668 [89] |_ 224.056 [99] |_ Latency distribution: 10% in 39.5655 secs 25% in 45.9014 secs 50% in 52.6002 secs 75% in 59.6084 secs 90% in 81.8000 secs 95% in 133.0850 secs 99% in 207.5297 secs Details (average, fastest, slowest): DNS+dialup: 46.6857 secs, 0.1815 secs, 224.0558 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0004 secs, 0.0000 secs, 0.0401 secs resp wait: 11.7872 secs, 0.0891 secs, 178.0161 secs resp read: 0.0021 secs, 0.0001 secs, 0.5959 secs Status code distribution: [200] 8780 responses [502] 33 responses [503] 101 responses Error distribution: [1065] Get "https://www.electrobsd.org/": context deadline exceeded [21] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/ Summary: Total: 341.1197 secs Slowest: 112.2543 secs Fastest: 0.1199 secs Average: 30.2646 secs Requests/sec: 29.3152 Total data: 302505 bytes Size/request: 30 bytes Response time histogram: 0.120 [1] | 11.333 [231] |_ 22.547 [762] |____ 33.760 [7305] |________________________________________ 44.974 [657] |____ 56.187 [598] |___ 67.401 [229] |_ 78.614 [155] |_ 89.827 [4] | 101.041 [0] | 112.254 [9] | Latency distribution: 10% in 22.5617 secs 25% in 25.1543 secs 50% in 27.8003 secs 75% in 30.8988 secs 90% in 45.0318 secs 95% in 55.3919 secs 99% in 68.8072 secs Details (average, fastest, slowest): DNS+dialup: 24.5586 secs, 0.1199 secs, 112.2543 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.0010 secs, 0.0000 secs, 0.0764 secs resp wait: 5.6970 secs, 0.0946 secs, 46.1816 secs resp read: 0.0079 secs, 0.0001 secs, 0.9150 secs Status code distribution: [200] 9908 responses [503] 43 responses Error distribution: [12] Get "https://www.electrobsd.org/": context deadline exceeded [37] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
[fk@benchmark-vm ~]$ hey -x http://172.16.1.7:8118 -n 10000 -c 1000 -disable-keepalive -t 0 https://www.electrobsd.org/ Summary: Total: 843.2849 secs Slowest: 181.8306 secs Fastest: 1.4541 secs Average: 81.1763 secs Requests/sec: 11.8584 Total data: 211050 bytes Size/request: 40 bytes Response time histogram: 1.454 [1] | 19.492 [97] |___ 37.529 [162] |____ 55.567 [541] |_______________ 73.605 [1447] |________________________________________ 91.642 [1223] |__________________________________ 109.680 [831] |_______________________ 127.718 [512] |______________ 145.755 [265] |_______ 163.793 [75] |__ 181.831 [6] | Latency distribution: 10% in 47.1757 secs 25% in 64.4024 secs 50% in 77.7478 secs 75% in 99.0196 secs 90% in 116.5997 secs 95% in 133.8875 secs 99% in 149.5484 secs Details (average, fastest, slowest): DNS+dialup: 67.7622 secs, 1.4541 secs, 181.8306 secs DNS-lookup: 0.0000 secs, 0.0000 secs, 0.0000 secs req write: 0.2751 secs, 0.0000 secs, 4.6106 secs resp wait: 12.9506 secs, 0.1315 secs, 78.1341 secs resp read: 0.1851 secs, 0.0001 secs, 12.6156 secs Status code distribution: [200] 5130 responses [503] 30 responses Error distribution: [4734] Get "https://www.electrobsd.org/": context deadline exceeded [106] Get "https://www.electrobsd.org/": proxyconnect tcp: dial tcp 172.16.1.7:8118: connect: operation timed out
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 6:43 | 319M |
| LibreSSL 3.2.3_2 | 6:45 | 347M |
| LibreSSL devel 3.3.1 | 10:56 | 446M |
| OpenSSL 1.1.1j,1 | 6:56 | 319M |
| OpenSSL devel 3.0.0.a13 | 9:15 | 348M |
| wolfSSL 4.7.0_6 | 4:21 | 271M |
| mbedtls 2.16.10 | 11:21 | 260M |
I decided to try siege which is written in C to see if it also shows skewed results like the benchmarking tools written in go.
While siege doesn't seem to offer histograms it provides a Longest transaction value and as it's rather high it looks the go runtime wasn't responsible for the effect after all.
siege requires a configuration file. I modified the default configuration to use Privoxy as proxy, disabled JSON mode and disabled verbose mode (which results in an output line for each request made).
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege...siege aborted due to excessive socket failure; you can change the failure threshold in $HOME/.siegerc Transactions: 28372 hits Availability: 96.23 % Elapsed time: 938.55 secs Data transferred: 16.66 MB Response time: 30.21 secs Transaction rate: 30.23 trans/sec Throughput: 0.02 MB/sec Concurrency: 913.16 Successful transactions: 28372 Failed transactions: 1112 Longest transaction: 98.49 Shortest transaction: 0.09
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 29635 hits Availability: 98.78 % Elapsed time: 1314.94 secs Data transferred: 12.06 MB Response time: 41.17 secs Transaction rate: 22.54 trans/sec Throughput: 0.01 MB/sec Concurrency: 927.84 Successful transactions: 29635 Failed transactions: 365 Longest transaction: 88.52 Shortest transaction: 0.09
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 29722 hits Availability: 99.07 % Elapsed time: 1491.27 secs Data transferred: 11.50 MB Response time: 46.84 secs Transaction rate: 19.93 trans/sec Throughput: 0.01 MB/sec Concurrency: 933.50 Successful transactions: 29722 Failed transactions: 278 Longest transaction: 143.56 Shortest transaction: 0.09
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 29431 hits Availability: 98.10 % Elapsed time: 1217.58 secs Data transferred: 13.37 MB Response time: 36.14 secs Transaction rate: 24.17 trans/sec Throughput: 0.01 MB/sec Concurrency: 873.47 Successful transactions: 29430 Failed transactions: 569 Longest transaction: 398.73 Shortest transaction: 0.07
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 29696 hits Availability: 98.99 % Elapsed time: 1410.63 secs Data transferred: 11.67 MB Response time: 44.16 secs Transaction rate: 21.05 trans/sec Throughput: 0.01 MB/sec Concurrency: 929.60 Successful transactions: 29696 Failed transactions: 304 Longest transaction: 100.67 Shortest transaction: 0.10
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege...siege aborted due to excessive socket failure; you can change the failure threshold in $HOME/.siegerc Transactions: 24798 hits Availability: 94.48 % Elapsed time: 1108.23 secs Data transferred: 17.74 MB Response time: 24.02 secs Transaction rate: 22.38 trans/sec Throughput: 0.02 MB/sec Concurrency: 537.52 Successful transactions: 24233 Failed transactions: 1448 Longest transaction: 426.51 Shortest transaction: 0.18
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=30 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 29957 hits Availability: 99.86 % Elapsed time: 3575.75 secs Data transferred: 10.00 MB Response time: 115.71 secs Transaction rate: 8.38 trans/sec Throughput: 0.00 MB/sec Concurrency: 969.42 Successful transactions: 29957 Failed transactions: 43 Longest transaction: 165.54 Shortest transaction: 0.20
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 13:50 | 501M |
| LibreSSL 3.2.3_2 | 19:35 | 478M |
| LibreSSL devel 3.3.1 | 22:24 | 527M |
| OpenSSL 1.1.1j,1 | 17:11 | 554M |
| OpenSSL devel 3.0.0.a13 | 20:50 | 372M |
| wolfSSL 4.7.0_6 | 10:11 | 515M |
| mbedtls 2.16.10 | 56:19 | 96720K |
All Privoxy ports have been updated, an experimental patch has been added that allows to change the scheduling policy using sched_setscheduler.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9960 hits Availability: 99.60 % Elapsed time: 416.73 secs Data transferred: 3.50 MB Response time: 29.56 secs Transaction rate: 23.90 trans/sec Throughput: 0.01 MB/sec Concurrency: 706.39 Successful transactions: 9960 Failed transactions: 40 Longest transaction: 311.88 Shortest transaction: 0.09
While the test was running the ssh session to the Privoxy VM became unresponsive for several minutes, top showed a nice value of r31F most of the time. Given that and the 311.88 seconds it supposedly took to handle the longest transaction, SCHED_FIFO doesn't appear to be useful for Privoxy.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out Transactions: 9880 hits Availability: 98.80 % Elapsed time: 377.66 secs Data transferred: 3.89 MB Response time: 29.81 secs Transaction rate: 26.16 trans/sec Throughput: 0.01 MB/sec Concurrency: 779.75 Successful transactions: 9880 Failed transactions: 120 Longest transaction: 89.09 Shortest transaction: 0.09
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9603 hits Availability: 96.03 % Elapsed time: 601.98 secs Data transferred: 5.89 MB Response time: 29.62 secs Transaction rate: 15.95 trans/sec Throughput: 0.01 MB/sec Concurrency: 472.53 Successful transactions: 9603 Failed transactions: 397 Longest transaction: 388.54 Shortest transaction: 0.08
Enabling SCHED_RR resulted in a temporary unresponsive ssh session as well. For the last minutes of the test top showed that Privoxy was using three threads, a nice value of r31 and was idle.
The Privoxy log confirmed that Privoxy didn't do any work for about four minutes near the end of the test:
15:15:52.791 818984200 Crunch: Connection failure: https://www.electrobsd.org/ 172.16.1.6 - - [18/Mar/2021:15:15:52 +0100] "GET https://www.electrobsd.org/ HTTP/1.1" 503 7035 15:19:37.838 81787f300 Error: The TLS/SSL handshake with the server failed: no TLS/SSL errors detected
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9965 hits Availability: 99.65 % Elapsed time: 369.88 secs Data transferred: 3.47 MB Response time: 28.65 secs Transaction rate: 26.94 trans/sec Throughput: 0.01 MB/sec Concurrency: 771.94 Successful transactions: 9965 Failed transactions: 35 Longest transaction: 71.43 Shortest transaction: 0.08
This test was expected to get similar results as the one with scheduling policy 2 and it did.
As both non-default scheduling policies seem to be worse than the default, polishing the scheduling policy patch probably isn't worth it.
I've patched siege to allow https requests over a proxy using keep-alive
and enabled keep-alive in the siege configuration file, ignoring the
TRIPLE CAUTION: We don't recommend you set this to keep-alive
comment.
One obvious issue is that the concurrency seems to vary significantly. Something to investigate later on.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 97.60 secs Data transferred: 3.24 MB Response time: 0.12 secs Transaction rate: 102.46 trans/sec Throughput: 0.03 MB/sec Concurrency: 12.32 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 5.29 Shortest transaction: 0.01
The Privoxy log confirms that 1000 connections where opened to send 10000 requests but as mentioned before the concurrency is quite a bit lower than requested.
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --statistics /var/log/privoxy/privoxy.log
Client requests total: 10000
No crunches detected. Is 'debug 1024' enabled?
Server keep-alive offers: 10000 (100.00%)
New outgoing connections: 1000 (10.00%)
Reused server connections: 9000 (90.00%; server offers accepted: 90.00%)
Empty responses: 0 (0.00%)
Empty responses on new connections: 0 (0.00%)
Empty responses on reused connections: 0 (0.00%)
Client connections: 1000
Bytes of content transfered to the client: 3520000
Client requests per connection distribution:
1000: 10
0: 1
Enable --show-complete-request-distribution to get less common numbers as well.
Improperly accounted requests: ~0
Method distribution:
10000 : GET
Client HTTP versions:
10000 : HTTP/1.1
HTTP status codes:
10000 : 200
URL statistics are disabled. Increase --url-statistics-threshold to enable them.
Passed request statistics are disabled. Increase --passed-request-statistics-threshold to enable them.
Host statistics are disabled. Increase --host-statistics-threshold to enable them.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 99.43 secs Data transferred: 3.24 MB Response time: 0.16 secs Transaction rate: 100.57 trans/sec Throughput: 0.03 MB/sec Concurrency: 16.35 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 6.20 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 90.48 secs Data transferred: 3.24 MB Response time: 0.11 secs Transaction rate: 110.53 trans/sec Throughput: 0.04 MB/sec Concurrency: 11.70 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 5.48 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9999 hits Availability: 99.99 % Elapsed time: 105.88 secs Data transferred: 3.24 MB Response time: 0.87 secs Transaction rate: 94.44 trans/sec Throughput: 0.03 MB/sec Concurrency: 82.33 Successful transactions: 9999 Failed transactions: 1 Longest transaction: 19.94 Shortest transaction: 0.01
It's interesting tha the Concurrency went up compared to the previous tests.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 92.08 secs Data transferred: 3.24 MB Response time: 0.16 secs Transaction rate: 108.60 trans/sec Throughput: 0.04 MB/sec Concurrency: 17.70 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 5.52 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 90.33 secs Data transferred: 3.24 MB Response time: 0.36 secs Transaction rate: 110.71 trans/sec Throughput: 0.04 MB/sec Concurrency: 39.46 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 11.46 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9956 hits Availability: 99.56 % Elapsed time: 158.23 secs Data transferred: 3.23 MB Response time: 6.84 secs Transaction rate: 62.92 trans/sec Throughput: 0.02 MB/sec Concurrency: 430.35 Successful transactions: 9956 Failed transactions: 44 Longest transaction: 118.43 Shortest transaction: 0.01
The concurrency went up to 430!
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 0:59 | 99M |
| LibreSSL 3.2.3_2 | 1:15 | 66284K |
| LibreSSL devel 3.3.1 | 1:12 | 44536K |
| OpenSSL 1.1.1j,1 | 1:14 | 209M |
| OpenSSL devel 3.0.0.a13 | 1:14 | 91876K |
| wolfSSL 4.7.0_6 | 0:54 | 108M |
| mbedtls 2.16.10 | 2:24 | 181M |
The LibreSSL port has been updated to 3.2.5.
The Privoxy ports haven been updated as well. The Privoxy's wolfSSL-specific code now contains an experimental patch for testing purposes that lets Privoxy use a single SSL context for the server connections instead of setting up a unique context for each connection.
For this test siege is configured with "connection = close" again.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9949 hits Availability: 99.49 % Elapsed time: 366.41 secs Data transferred: 3.57 MB Response time: 28.11 secs Transaction rate: 27.15 trans/sec Throughput: 0.01 MB/sec Concurrency: 763.33 Successful transactions: 9949 Failed transactions: 51 Longest transaction: 66.06 Shortest transaction: 0.09
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9902 hits Availability: 99.02 % Elapsed time: 803.27 secs Data transferred: 3.87 MB Response time: 40.86 secs Transaction rate: 12.33 trans/sec Throughput: 0.00 MB/sec Concurrency: 503.71 Successful transactions: 9902 Failed transactions: 98 Longest transaction: 405.18 Shortest transaction: 0.14
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9936 hits Availability: 99.36 % Elapsed time: 496.41 secs Data transferred: 3.65 MB Response time: 40.42 secs Transaction rate: 20.02 trans/sec Throughput: 0.01 MB/sec Concurrency: 808.93 Successful transactions: 9936 Failed transactions: 64 Longest transaction: 103.87 Shortest transaction: 0.09
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9825 hits Availability: 98.25 % Elapsed time: 414.31 secs Data transferred: 4.36 MB Response time: 33.77 secs Transaction rate: 23.71 trans/sec Throughput: 0.01 MB/sec Concurrency: 800.78 Successful transactions: 9825 Failed transactions: 175 Longest transaction: 86.27 Shortest transaction: 0.09
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out Transactions: 9918 hits Availability: 99.18 % Elapsed time: 556.85 secs Data transferred: 3.76 MB Response time: 47.52 secs Transaction rate: 17.81 trans/sec Throughput: 0.01 MB/sec Concurrency: 846.32 Successful transactions: 9918 Failed transactions: 82 Longest transaction: 101.36 Shortest transaction: 0.28
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9992 hits Availability: 99.92 % Elapsed time: 873.68 secs Data transferred: 3.28 MB Response time: 29.50 secs Transaction rate: 11.44 trans/sec Throughput: 0.00 MB/sec Concurrency: 337.39 Successful transactions: 9583 Failed transactions: 8 Longest transaction: 414.19 Shortest transaction: 0.06
It looks like my experimental wolfSSL-specific patch introduces a problem. Both siege and Privoxy where idle for longer amounts of time while siege was running.
Apparently Privoxy did not do log-worthy things for nearly five minutes:
2021-03-21 11:12:46.383 8118ab900 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-21 11:12:46.383 8118ab900 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2 2021-03-21 11:17:38.971 822340500 Connect: Failed to shutdown server connection on socket 70. Attempts so far: 2, ret: 2 2021-03-21 11:17:38.972 822340500 Connect: Not shutting down server connection on socket 70. The socket is no longer alive. 2021-03-21 11:17:38.972 822340500 Connect: Closing client socket 69. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1.
Update: The problem was reproducible:
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9973 hits Availability: 99.73 % Elapsed time: 891.48 secs Data transferred: 3.41 MB Response time: 22.36 secs Transaction rate: 11.19 trans/sec Throughput: 0.00 MB/sec Concurrency: 250.14 Successful transactions: 9669 Failed transactions: 27 Longest transaction: 368.88 Shortest transaction: 0.06
Again it looks like Privoxy spent about five minutes trying to shut down a connection:
2021-03-21 16:23:08.203 810bfaf00 Connect: Not shutting down server connection on socket 638. The socket is no longer alive. 2021-03-21 16:23:08.204 810bfaf00 Connect: Closing client socket 57. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-21 16:23:08.204 810bfaf00 Connect: Failed to shutdown client connection on socket 57. Attempts so far: 1, ret: 2 2021-03-21 16:27:24.871 803a69e00 Connect: Failed to shutdown server connection on socket 1780. Attempts so far: 2, ret: 2 2021-03-21 16:27:24.872 803a69e00 Connect: Not shutting down server connection on socket 1780. The socket is no longer alive. 2021-03-21 16:27:24.872 803a69e00 Connect: Closing client socket 628. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-21 16:27:24.872 803a69e00 Connect: Failed to shutdown client connection on socket 628. Attempts so far: 1, ret: 2
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9989 hits Availability: 99.89 % Elapsed time: 1244.04 secs Data transferred: 3.31 MB Response time: 113.31 secs Transaction rate: 8.03 trans/sec Throughput: 0.00 MB/sec Concurrency: 909.86 Successful transactions: 9989 Failed transactions: 11 Longest transaction: 168.69 Shortest transaction: 1.75
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 4:56 | 400M |
| LibreSSL 3.2.5 | 7:08 | 443M |
| LibreSSL devel 3.3.1 | 7:03 | 335M |
| OpenSSL 1.1.1j,1 | 5:42 | 345M |
| OpenSSL devel 3.0.0.a13 | 7:55 | 324M |
| wolfSSL 4.7.0_6 test 1 | 3:26 | 541M |
| wolfSSL 4.7.0_6 test 2 | 3:31 | 535M |
| mbedtls 2.16.10 | 18:53 | 131M |
The siege configuration remains the same but the requested concurrency is reduced to 10 while the reps are set to 1000 to get a similar amount of requests as before.
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 9956 hits Availability: 99.56 % Elapsed time: 580.95 secs Data transferred: 3.52 MB Response time: 0.46 secs Transaction rate: 17.14 trans/sec Throughput: 0.01 MB/sec Concurrency: 7.93 Successful transactions: 9956 Failed transactions: 44 Longest transaction: 30.17 Shortest transaction: 0.07
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out Transactions: 9958 hits Availability: 99.58 % Elapsed time: 514.22 secs Data transferred: 3.50 MB Response time: 0.42 secs Transaction rate: 19.37 trans/sec Throughput: 0.01 MB/sec Concurrency: 8.07 Successful transactions: 9958 Failed transactions: 42 Longest transaction: 30.30 Shortest transaction: 0.07
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 9953 hits Availability: 99.53 % Elapsed time: 562.77 secs Data transferred: 3.54 MB Response time: 0.51 secs Transaction rate: 17.69 trans/sec Throughput: 0.01 MB/sec Concurrency: 9.04 Successful transactions: 9953 Failed transactions: 47 Longest transaction: 31.95 Shortest transaction: 0.08
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 9964 hits Availability: 99.64 % Elapsed time: 518.83 secs Data transferred: 3.47 MB Response time: 0.39 secs Transaction rate: 19.20 trans/sec Throughput: 0.01 MB/sec Concurrency: 7.43 Successful transactions: 9964 Failed transactions: 36 Longest transaction: 30.54 Shortest transaction: 0.07
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out Transactions: 9991 hits Availability: 99.91 % Elapsed time: 463.96 secs Data transferred: 3.29 MB Response time: 0.40 secs Transaction rate: 21.53 trans/sec Throughput: 0.01 MB/sec Concurrency: 8.61 Successful transactions: 9991 Failed transactions: 9 Longest transaction: 30.19 Shortest transaction: 0.08
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 9973 hits Availability: 99.73 % Elapsed time: 416.36 secs Data transferred: 3.41 MB Response time: 0.32 secs Transaction rate: 23.95 trans/sec Throughput: 0.01 MB/sec Concurrency: 7.58 Successful transactions: 9973 Failed transactions: 27 Longest transaction: 30.10 Shortest transaction: 0.05
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 9999 hits Availability: 99.99 % Elapsed time: 1154.95 secs Data transferred: 3.25 MB Response time: 1.14 secs Transaction rate: 8.66 trans/sec Throughput: 0.00 MB/sec Concurrency: 9.87 Successful transactions: 9999 Failed transactions: 1 Longest transaction: 30.54 Shortest transaction: 0.18
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 4:23 | 71944K |
| LibreSSL 3.2.5 | 5:22 | 68748K |
| LibreSSL devel 3.3.1 | 5:55 | 62216K |
| OpenSSL 1.1.1j,1 | 4:52 | 85780K |
| OpenSSL devel 3.0.0.a13 | 6:08 | 70112K |
| wolfSSL 4.7.0_6 | 1:57 | 21224K |
| mbedtls 2.16.10 | 18:06 | 15432K |
The significantly reduced CPU time when using wolfSSL indicates that the wolfSSL-specific patch to share the SSL context may not be entirely useless.
I've updated the Privoxy ports to include a wolfSSL-specific patch that
sets MAX_SHUTDOWN_ATTEMPTS to 1. The effect is that wolfSSL_shutdown() is
not called more than once, even if it returns WOLFSSL_SHUTDOWN_NOT_DONE
in which case one is supposed to call wolfSSL_shutdown again to complete
according to wolfssl/ssl.h.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9964 hits Availability: 99.64 % Elapsed time: 532.43 secs Data transferred: 3.46 MB Response time: 28.14 secs Transaction rate: 18.71 trans/sec Throughput: 0.01 MB/sec Concurrency: 526.54 Successful transactions: 9493 Failed transactions: 36 Longest transaction: 352.60 Shortest transaction: 0.06
Not retrying shutdowns doesn't seem to work around the issue:
2021-03-21 17:35:47.531 80abc7500 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out. 2021-03-21 17:35:47.532 80abc7500 Crunch: Connection failure: https://www.electrobsd.org/ 172.16.1.6 - - [21/Mar/2021:17:35:47 +0100] "GET https://www.electrobsd.org/ HTTP/1.1" 503 7035 2021-03-21 17:35:47.533 80abc7500 Connect: Failed to shutdown client connection on socket 59. Attempts so far: 1, ret: 2 2021-03-21 17:35:47.533 80abc7500 Error: Failed to shutdown client connection on socket 59 after 1 attempts. ret: 2, error: 0, unknown error number 2021-03-21 17:38:09.441 812906b00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-21 17:38:09.442 812906b00 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
| TLS library | CPU time | Memory size |
|---|---|---|
| wolfSSL 4.7.0_6 | 3:05 | 581M |
The Privoxy ports have been updated. They now contain a Privoxy-Log-Parser patch to detect periods of inactivity.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9963 hits Availability: 99.63 % Elapsed time: 672.10 secs Data transferred: 3.47 MB Response time: 28.66 secs Transaction rate: 14.82 trans/sec Throughput: 0.01 MB/sec Concurrency: 424.91 Successful transactions: 9719 Failed transactions: 37 Longest transaction: 332.53 Shortest transaction: 0.06
Privoxy-Log-Parser detects inactivity:
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210322-wolfssl-c1000-cc.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 | grep -B 3 -A 2 '^Detected inactivity' 2021-03-22 07:12:20.184 80d962c00 Crunch: Connection failure: https://www.electrobsd.org/ 2021-03-22 07:12:20.185 80d962c00 Connect: Failed to shutdown client connection on socket 6. Attempts so far: 1, ret: 2 2021-03-22 07:12:20.185 80d962c00 Error: Failed to shutdown client connection on socket 6 after 1 attempts. ret: 2, error: 0, unknown error number Detected inactivity: 96305 msecs 2021-03-22 07:13:56.490 80fdc9600 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-22 07:13:56.491 80fdc9600 Connect: Closing client socket 906. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
| TLS library | CPU time | Memory size |
|---|---|---|
| wolfSSL 4.7.0_6 | 4:59 | 511M |
The Privoxy ports have been updated to remove the wolfSSL-specific ctx-reuse hack which requires additional work before it's ready.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9964 hits Availability: 99.64 % Elapsed time: 384.18 secs Data transferred: 3.47 MB Response time: 29.86 secs Transaction rate: 25.94 trans/sec Throughput: 0.01 MB/sec Concurrency: 774.41 Successful transactions: 9964 Failed transactions: 36 Longest transaction: 60.51 Shortest transaction: 0.09
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 /var/log/privoxy/privoxy.log | grep -B 3 -A 2 '^Detected inactivity'
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9935 hits Availability: 99.35 % Elapsed time: 464.12 secs Data transferred: 3.66 MB Response time: 37.46 secs Transaction rate: 21.41 trans/sec Throughput: 0.01 MB/sec Concurrency: 801.86 Successful transactions: 9935 Failed transactions: 65 Longest transaction: 87.09 Shortest transaction: 0.33
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 /var/log/privoxy/privoxy.log | grep -B 3 -A 2 '^Detected inactivity'
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9954 hits Availability: 99.54 % Elapsed time: 483.00 secs Data transferred: 3.54 MB Response time: 41.74 secs Transaction rate: 20.61 trans/sec Throughput: 0.01 MB/sec Concurrency: 860.30 Successful transactions: 9954 Failed transactions: 46 Longest transaction: 89.28 Shortest transaction: 0.36
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 /var/log/privoxy/privoxy.log | grep -B 3 -A 2 '^Detected inactivity'
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 /var/log/privoxy/privoxy.log | grep -B 3 -A 2 '^Detected inactivity'
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9944 hits Availability: 99.44 % Elapsed time: 422.99 secs Data transferred: 3.60 MB Response time: 33.64 secs Transaction rate: 23.51 trans/sec Throughput: 0.01 MB/sec Concurrency: 790.90 Successful transactions: 9944 Failed transactions: 56 Longest transaction: 77.73 Shortest transaction: 0.11
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9892 hits Availability: 98.92 % Elapsed time: 589.88 secs Data transferred: 3.94 MB Response time: 50.67 secs Transaction rate: 16.77 trans/sec Throughput: 0.01 MB/sec Concurrency: 849.68 Successful transactions: 9892 Failed transactions: 108 Longest transaction: 166.45 Shortest transaction: 0.20
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 | grep -B 3 -A 2 '^Detected inactivity'
I've added the sort command as there were log messages written out of the chronological order which caused false positives when parsing the unsorted file.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9985 hits Availability: 99.85 % Elapsed time: 577.09 secs Data transferred: 3.33 MB Response time: 26.49 secs Transaction rate: 17.30 trans/sec Throughput: 0.01 MB/sec Concurrency: 458.35 Successful transactions: 9756 Failed transactions: 15 Longest transaction: 443.84 Shortest transaction: 0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 | grep -B 3 -A 2 '^Detected inactivity' 2021-03-22 09:20:19.394 812728f00 Crunch: Connection failure: https://www.electrobsd.org/ 2021-03-22 09:20:19.395 812728f00 Connect: Failed to shutdown client connection on socket 129. Attempts so far: 1, ret: 2 2021-03-22 09:20:19.395 812728f00 Error: Failed to shutdown client connection on socket 129 after 1 attempts. ret: 2, error: 0, unknown error number Detected inactivity: 156418 msecs 2021-03-22 09:22:55.813 8083d4f00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-22 09:22:55.814 8083d4f00 Connect: Closing client socket 1232. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
Looks like the issue isn't actually caused by the wolfSSL-specific ctx-reuse hack which is no longer present.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9991 hits Availability: 99.91 % Elapsed time: 1188.16 secs Data transferred: 3.30 MB Response time: 108.66 secs Transaction rate: 8.41 trans/sec Throughput: 0.00 MB/sec Concurrency: 913.69 Successful transactions: 9991 Failed transactions: 9 Longest transaction: 153.42 Shortest transaction: 0.70
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 30000 | grep -B 3 -A 2 '^Detected inactivity'
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 5:14 | 344M |
| LibreSSL 3.2.5 | 6:26 | 345M |
| LibreSSL devel 3.3.1 | 7:04 | 366M |
| OpenSSL 1.1.1j,1 | 5:45 | 364M |
| OpenSSL devel 3.0.0.a13 | 8:15 | 453M |
| wolfSSL 4.7.0_6 | 4:00 | 460M |
| mbedtls 2.16.10 | 18:31 | 95860K |
In the siege configuration I've disabled the parser mode which may or may not speed up siege. The ElectroBSD homepage doesn't use external images or CSS files so the number of requests shouldn't be affected.
The mbedTLS port has been updated to remove a patch coming from FreeBSD that adds "DTLS-SRTP (RFC 5764)" support. Privoxy isn't using DTLS but it was worth a try. The privoxy-experimental-mbedtls port was recompiled due to the dependency change.
The siege concurrency has been reduced to 1 and the repetitions set to 1000 (so the number of requests is reduced to ~1000). While the tests were running the systems were mostly idle, presumably waiting for the network.
It looks like siege has socket management issues that should be investigated further.
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1 concurrent users for battle. The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out Transactions: 999 hits Availability: 99.90 % Elapsed time: 178.88 secs Data transferred: 0.32 MB Response time: 0.15 secs Transaction rate: 5.58 trans/sec Throughput: 0.00 MB/sec Concurrency: 0.82 Successful transactions: 999 Failed transactions: 1 Longest transaction: 15.92 Shortest transaction: 0.07
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1 concurrent users for battle. The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out Transactions: 998 hits Availability: 99.80 % Elapsed time: 220.91 secs Data transferred: 0.32 MB Response time: 0.16 secs Transaction rate: 4.52 trans/sec Throughput: 0.00 MB/sec Concurrency: 0.72 Successful transactions: 998 Failed transactions: 2 Longest transaction: 6.31 Shortest transaction: 0.07
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 999 hits Availability: 99.90 % Elapsed time: 160.31 secs Data transferred: 0.33 MB Response time: 0.16 secs Transaction rate: 6.23 trans/sec Throughput: 0.00 MB/sec Concurrency: 0.98 Successful transactions: 999 Failed transactions: 1 Longest transaction: 30.07 Shortest transaction: 0.08
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1 concurrent users for battle. The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out [alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out Transactions: 998 hits Availability: 99.80 % Elapsed time: 200.48 secs Data transferred: 0.32 MB Response time: 0.14 secs Transaction rate: 4.98 trans/sec Throughput: 0.00 MB/sec Concurrency: 0.69 Successful transactions: 998 Failed transactions: 2 Longest transaction: 1.13 Shortest transaction: 0.07
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 134.23 secs Data transferred: 0.32 MB Response time: 0.13 secs Transaction rate: 7.45 trans/sec Throughput: 0.00 MB/sec Concurrency: 0.98 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 0.61 Shortest transaction: 0.08
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 124.91 secs Data transferred: 0.32 MB Response time: 0.12 secs Transaction rate: 8.01 trans/sec Throughput: 0.00 MB/sec Concurrency: 1.00 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 5.10 Shortest transaction: 0.06
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 248.80 secs Data transferred: 0.32 MB Response time: 0.25 secs Transaction rate: 4.02 trans/sec Throughput: 0.00 MB/sec Concurrency: 1.00 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 3.54 Shortest transaction: 0.16
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 0:25 | 25768K |
| LibreSSL 3.2.5 | 0:31 | 20312K |
| LibreSSL devel 3.3.1 | 0:34 | 21620K |
| OpenSSL 1.1.1j,1 | 0:27 | 26516K |
| OpenSSL devel 3.0.0.a13 | 0:36 | 25756K |
| wolfSSL 4.7.0_6 | 0:16 | 8420K |
| mbedtls 2.16.10_1 | 1:47 | 7852K |
The Privoxy ports have been updated. The wolfSSL-specific ctx-reuse hack is back but hidden behind a configuration directive so it can be enabled at runtime.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9962 hits Availability: 99.62 % Elapsed time: 372.49 secs Data transferred: 3.49 MB Response time: 28.29 secs Transaction rate: 26.74 trans/sec Throughput: 0.01 MB/sec Concurrency: 756.72 Successful transactions: 9962 Failed transactions: 38 Longest transaction: 70.66 Shortest transaction: 0.08
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9912 hits Availability: 99.12 % Elapsed time: 448.02 secs Data transferred: 3.80 MB Response time: 36.20 secs Transaction rate: 22.12 trans/sec Throughput: 0.01 MB/sec Concurrency: 800.97 Successful transactions: 9912 Failed transactions: 88 Longest transaction: 78.40 Shortest transaction: 0.08
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9939 hits Availability: 99.39 % Elapsed time: 492.29 secs Data transferred: 3.63 MB Response time: 40.01 secs Transaction rate: 20.19 trans/sec Throughput: 0.01 MB/sec Concurrency: 807.72 Successful transactions: 9939 Failed transactions: 61 Longest transaction: 80.22 Shortest transaction: 0.12
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9903 hits Availability: 99.03 % Elapsed time: 481.68 secs Data transferred: 3.86 MB Response time: 37.87 secs Transaction rate: 20.56 trans/sec Throughput: 0.01 MB/sec Concurrency: 778.49 Successful transactions: 9903 Failed transactions: 97 Longest transaction: 115.86 Shortest transaction: 0.12
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9958 hits Availability: 99.58 % Elapsed time: 511.93 secs Data transferred: 3.51 MB Response time: 42.92 secs Transaction rate: 19.45 trans/sec Throughput: 0.01 MB/sec Concurrency: 834.92 Successful transactions: 9958 Failed transactions: 42 Longest transaction: 92.80 Shortest transaction: 0.09
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9967 hits Availability: 99.67 % Elapsed time: 514.52 secs Data transferred: 3.45 MB Response time: 26.77 secs Transaction rate: 19.37 trans/sec Throughput: 0.01 MB/sec Concurrency: 518.50 Successful transactions: 9899 Failed transactions: 33 Longest transaction: 411.27 Shortest transaction: 0.07
Unfortunately I failed to run Privoxy-Log-Parser to investigate the periods of inactivity that occurred. I've repeated the test out of order.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out Transactions: 9928 hits Availability: 99.28 % Elapsed time: 563.20 secs Data transferred: 3.69 MB Response time: 25.84 secs Transaction rate: 17.63 trans/sec Throughput: 0.01 MB/sec Concurrency: 455.42 Successful transactions: 9784 Failed transactions: 72 Longest transaction: 411.16 Shortest transaction: 0.08
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -B 3 -A 2 '^Detected inactivity' 2021-03-23 09:18:43.600 82bd00f00 Connect: Failed to shutdown client connection on socket 26. Attempts so far: 1, ret: 2 2021-03-23 09:18:43.600 82bd00f00 Crunch: Connection failure: https://www.electrobsd.org/ 2021-03-23 09:18:43.600 82bd00f00 Error: Failed to shutdown client connection on socket 26 after 1 attempts. ret: 2, error: 0, unknown error number Detected inactivity: 12798 msecs 2021-03-23 09:18:56.398 806d7ed00 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out. 2021-03-23 09:18:56.400 806d7ed00 Connect: Failed to shutdown client connection on socket 121. Attempts so far: 1, ret: 2 -- 2021-03-23 09:18:57.750 821f76400 Connect: Failed to shutdown client connection on socket 162. Attempts so far: 1, ret: 2 2021-03-23 09:18:57.750 821f76400 Crunch: Connection failure: https://www.electrobsd.org/ 2021-03-23 09:18:57.750 821f76400 Error: Failed to shutdown client connection on socket 162 after 1 attempts. ret: 2, error: 0, unknown error number Detected inactivity: 157780 msecs 2021-03-23 09:21:35.530 822afd300 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-23 09:21:35.531 822afd300 Connect: Closing client socket 1336. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. -- 2021-03-23 09:21:52.642 8054c5600 Connect: Failed to shutdown server connection on socket 9. Attempts so far: 1, ret: 2 2021-03-23 09:21:52.642 8054c5600 Error: Failed to shutdown client connection on socket 8 after 1 attempts. ret: 2, error: 0, unknown error number 2021-03-23 09:21:52.642 8054c5600 Error: Failed to shutdown server connection on socket 9 after 1 attempts. ret: 2, error: 0, unknown error number Detected inactivity: 14096 msecs 2021-03-23 09:22:06.738 81a8a2e00 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out. 2021-03-23 09:22:06.740 81a8a2e00 Crunch: Connection failure: https://www.electrobsd.org/ -- 2021-03-23 09:22:53.093 82c91b900 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2 2021-03-23 09:22:53.093 82c91b900 Error: Failed to shutdown client connection on socket 5 after 1 attempts. ret: 2, error: 0, unknown error number 2021-03-23 09:22:53.093 82c91b900 Error: Failed to shutdown server connection on socket 6 after 1 attempts. ret: 2, error: 0, unknown error number Detected inactivity: 10075 msecs 2021-03-23 09:23:03.168 82d69c300 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out. 2021-03-23 09:23:03.170 82d69c300 Connect: Failed to shutdown client connection on socket 25. Attempts so far: 1, ret: 2
[fk@privoxy-vm ~]$ grep -1000 '2021-03-23 09:21:35.530 822afd300 Error: X509 certificate' /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -B 3 -A 2 '^Detected inactivity' 2021-03-23 09:18:43.600 82bd00f00 Crunch: Connection failure: https://www.electrobsd.org/ 2021-03-23 09:18:43.600 82bd00f00 Connect: Failed to shutdown client connection on socket 26. Attempts so far: 1, ret: 2 2021-03-23 09:18:43.600 82bd00f00 Error: Failed to shutdown client connection on socket 26 after 1 attempts. ret: 2, error: 0, unknown error number Detected inactivity: 12798 msecs 2021-03-23 09:18:56.398 806d7ed00 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out. 2021-03-23 09:18:56.400 806d7ed00 Crunch: Connection failure: https://www.electrobsd.org/ -- 2021-03-23 09:18:57.750 821f76400 Crunch: Connection failure: https://www.electrobsd.org/ 2021-03-23 09:18:57.750 821f76400 Connect: Failed to shutdown client connection on socket 162. Attempts so far: 1, ret: 2 2021-03-23 09:18:57.750 821f76400 Error: Failed to shutdown client connection on socket 162 after 1 attempts. ret: 2, error: 0, unknown error number Detected inactivity: 157780 msecs 2021-03-23 09:21:35.530 822afd300 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-23 09:21:35.531 822afd300 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9951 hits Availability: 99.51 % Elapsed time: 579.41 secs Data transferred: 3.54 MB Response time: 28.32 secs Transaction rate: 17.17 trans/sec Throughput: 0.01 MB/sec Concurrency: 486.46 Successful transactions: 9552 Failed transactions: 49 Longest transaction: 427.17 Shortest transaction: 0.05
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -B 3 -A 2 '^Detected inactivity' 2021-03-23 08:32:10.885 804735900 Header: scan: Transfer-Encoding: chunked 2021-03-23 08:32:10.886 804735900 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2 2021-03-23 08:32:10.886 804735900 Error: Failed to shutdown client connection on socket 5 after 1 attempts. ret: 2, error: 0, unknown error number Detected inactivity: 11460 msecs 2021-03-23 08:32:22.346 81034f300 Connect: Connected to www.electrobsd.org[95.211.138.51]:443. 2021-03-23 08:32:22.346 81034f300 Connect: Created new connection to www.electrobsd.org:443 on socket 731. -- 2021-03-23 08:32:27.400 8253c8300 Crunch: Connection failure: https://www.electrobsd.org/ 2021-03-23 08:32:27.401 8253c8300 Connect: Failed to shutdown client connection on socket 230. Attempts so far: 1, ret: 2 2021-03-23 08:32:27.401 8253c8300 Error: Failed to shutdown client connection on socket 230 after 1 attempts. ret: 2, error: 0, unknown error number Detected inactivity: 147648 msecs 2021-03-23 08:34:55.049 8142e5d00 Connect: Failed to shutdown client connection on socket 477. Attempts so far: 1, ret: 2 2021-03-23 08:34:55.049 8142e5d00 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/ -- 2021-03-23 08:36:58.522 816aab500 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-23 08:36:58.522 816aab500 Header: scan: Server: nginx 2021-03-23 08:36:58.522 816aab500 Header: scan: Transfer-Encoding: chunked Detected inactivity: 34004 msecs 2021-03-23 08:37:32.526 8094bd700 Connect: Closing client socket 917. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-23 08:37:32.526 8094bd700 Connect: Closing server socket 1017 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5.
Due to the sorting the messages with the same time stamp and thread id may appear out of chronological order! Unfortunately the log file is already overwritten.
The unknown error number
is the result of the
unpolished patch to only call wolfSSL_shutdown() once
which the port still contains. I'll remove it in the next update as it doesn't
seem to work around the problem.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9998 hits Availability: 99.98 % Elapsed time: 1190.16 secs Data transferred: 3.26 MB Response time: 108.83 secs Transaction rate: 8.40 trans/sec Throughput: 0.00 MB/sec Concurrency: 914.25 Successful transactions: 9998 Failed transactions: 2 Longest transaction: 167.85 Shortest transaction: 0.18
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 5:08 | 382M |
| LibreSSL 3.2.5 | 6:09 | 360M |
| LibreSSL devel 3.3.1 | 6:48 | 319M |
| OpenSSL 1.1.1j,1 | 6:38 | 449M |
| OpenSSL devel 3.0.0.a13 | 7:11 | 300M |
| wolfSSL 4.7.0_6 (separate server WOLFSSL_CTX) 1 | 4:25 | 706M |
| wolfSSL 4.7.0_6 (separate server WOLFSSL_CTX) 2 | 4:19 | 840M |
| wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX) | 3:26 | 507M |
| mbedtls 2.16.10_1 | 18:28 | 99024K |
The Privoxy ports have been updated and no longer contain the wolfSSL-specific patch to only call wolfSSL_shutdown() once.
In the Privoxy VM the previously auto-tuned ZFS ARC size has been limited by setting the vfs.zfs.arc_max sysctl to 256 MB.
The siege port has been updated to include a patch to enable keep-alive mode on the command line but I have not yet investigated why the concurrency level isn't respected when using keep-alive. It looks like the performance of the system under test affects how many concurrent connections siege is using.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 91.76 secs Data transferred: 3.24 MB Response time: 0.06 secs Transaction rate: 108.98 trans/sec Throughput: 0.04 MB/sec Concurrency: 6.32 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 1.54 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 96.30 secs Data transferred: 3.24 MB Response time: 0.27 secs Transaction rate: 103.84 trans/sec Throughput: 0.03 MB/sec Concurrency: 28.25 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 7.87 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 91.98 secs Data transferred: 3.24 MB Response time: 0.96 secs Transaction rate: 108.72 trans/sec Throughput: 0.04 MB/sec Concurrency: 104.72 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 8.39 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 89.23 secs Data transferred: 3.24 MB Response time: 0.27 secs Transaction rate: 112.06 trans/sec Throughput: 0.04 MB/sec Concurrency: 30.07 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 6.01 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9996 hits Availability: 99.96 % Elapsed time: 102.48 secs Data transferred: 3.24 MB Response time: 1.47 secs Transaction rate: 97.54 trans/sec Throughput: 0.03 MB/sec Concurrency: 143.65 Successful transactions: 9996 Failed transactions: 4 Longest transaction: 26.71 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 84.31 secs Data transferred: 3.24 MB Response time: 0.06 secs Transaction rate: 118.61 trans/sec Throughput: 0.04 MB/sec Concurrency: 7.60 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 5.20 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 83.74 secs Data transferred: 3.24 MB Response time: 0.04 secs Transaction rate: 119.41 trans/sec Throughput: 0.04 MB/sec Concurrency: 4.69 Successful transactions: 10000 Failed transactions: 0 Longest transaction: 1.35 Shortest transaction: 0.01
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark --keep-alive "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9976 hits Availability: 99.76 % Elapsed time: 164.62 secs Data transferred: 3.23 MB Response time: 5.65 secs Transaction rate: 60.60 trans/sec Throughput: 0.02 MB/sec Concurrency: 342.47 Successful transactions: 9976 Failed transactions: 24 Longest transaction: 104.38 Shortest transaction: 0.01
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 0:59 | 34616K |
| LibreSSL 3.2.5 | 1:12 | 131M |
| LibreSSL devel 3.3.1 | 1:16 | 188M |
| OpenSSL 1.1.1j,1 | 1:06 | 150M |
| OpenSSL devel 3.0.0.a13 | 1:23 | 248M |
| wolfSSL 4.7.0_6 (separate server WOLFSSL_CTX) | 0:47 | 96380K |
| wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX) | 0:38 | 29192K |
| mbedtls 2.16.10_1 | 2:29 | 100M |
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9973 hits Availability: 99.73 % Elapsed time: 460.06 secs Data transferred: 3.42 MB Response time: 31.50 secs Transaction rate: 21.68 trans/sec Throughput: 0.01 MB/sec Concurrency: 682.93 Successful transactions: 9938 Failed transactions: 27 Longest transaction: 143.69 Shortest transaction: 0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -B 3 -A 2 '^Detected inactivity' 2021-03-23 11:17:08.071 80431a900 Header: scan: HTTP/1.1 200 OK 2021-03-23 11:17:08.071 80b4de200 Header: scan: HTTP/1.1 200 OK 2021-03-23 11:17:08.071 81f34df00 Header: scan: HTTP/1.1 200 OK Detected inactivity: 26176 msecs 2021-03-23 11:17:34.247 820518500 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out. Detected inactivity: 13587 msecs 2021-03-23 11:17:47.834 802c5bb00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-23 11:17:47.834 820674300 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). -- 2021-03-23 11:21:52.464 81d954100 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-23 11:21:52.464 81d954100 Header: scan: Server: nginx 2021-03-23 11:21:52.464 81d954100 Header: scan: Transfer-Encoding: chunked Detected inactivity: 31114 msecs 2021-03-23 11:22:23.578 804fe7400 Connect: Failed to shutdown server connection on socket 73. Attempts so far: 2, ret: 2 2021-03-23 11:22:23.578 804fe7400 Connect: Not shutting down server connection on socket 73. The socket is no longer alive. [fk@privoxy-vm ~]$ grep -1000 '2021-03-23 11:22:23.578 804fe7400 Connect: Failed' /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -B 3 -A 2 '^Detected inactivity' 2021-03-23 11:21:52.464 81d954100 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2 2021-03-23 11:21:52.464 81d954100 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-23 11:21:52.464 81d954100 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2 Detected inactivity: 31114 msecs 2021-03-23 11:22:23.578 804fe7400 Connect: Failed to shutdown server connection on socket 73. Attempts so far: 2, ret: 2 2021-03-23 11:22:23.578 804fe7400 Connect: Not shutting down server connection on socket 73. The socket is no longer alive.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9983 hits Availability: 99.83 % Elapsed time: 774.84 secs Data transferred: 3.34 MB Response time: 35.56 secs Transaction rate: 12.88 trans/sec Throughput: 0.00 MB/sec Concurrency: 458.17 Successful transactions: 9652 Failed transactions: 17 Longest transaction: 569.32 Shortest transaction: 0.05
Privoxy went idle several times.
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 100000 /var/log/privoxy/privoxy.log | grep -10 '^Detected inactivity' 2021-03-23 11:31:14.949 826cbe400 Header: scan: Transfer-Encoding: chunked 2021-03-23 11:31:14.949 826cbe400 Header: scan: Connection: close 2021-03-23 11:31:14.949 826cbe400 Header: scan: ETag: W/"5f429d52-1fe" 2021-03-23 11:31:14.949 826cbe400 Header: scan: Content-Encoding: gzip 2021-03-23 11:31:14.950 826cbe400 Connect: Looks like we got the last chunk together with the server headers. We better stop reading. 2021-03-23 11:31:14.950 826cbe400 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352. 2021-03-23 11:31:14.950 826cbe400 Connect: Closing server socket 78 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5. 2021-03-23 11:31:14.950 826cbe400 Connect: Failed to shutdown server connection on socket 78. Attempts so far: 1, ret: 2 2021-03-23 11:31:14.950 826cbe400 Connect: Closing client socket 39. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-23 11:31:14.950 826cbe400 Connect: Failed to shutdown client connection on socket 39. Attempts so far: 1, ret: 2 Detected inactivity: 115329 msecs 2021-03-23 11:33:10.279 80257c300 Connect: Failed to shutdown server connection on socket 141. Attempts so far: 2, ret: 2 2021-03-23 11:33:10.279 80257c300 Connect: Not shutting down server connection on socket 141. The socket is no longer alive. 2021-03-23 11:33:10.279 80257c300 Connect: Closing client socket 120. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-23 11:33:10.279 80257c300 Connect: Failed to shutdown client connection on socket 120. Attempts so far: 1, ret: 2 2021-03-23 11:33:10.531 802416000 Connect: Waiting for the next client connection. Currently active threads: 338 2021-03-23 11:33:10.531 82b25a900 Connect: Accepted connection from 172.16.1.6 on socket 5 2021-03-23 11:33:10.531 82b25a900 Header: scan: CONNECT www.electrobsd.org:443 HTTP/1.0 2021-03-23 11:33:10.531 82b25a900 Header: scan: User-agent: Proxy-User 2021-03-23 11:33:10.531 82b25a900 Header: addh-unique: Host: www.electrobsd.org:443 2021-03-23 11:33:10.532 82b25a900 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b
| TLS library | CPU time | Memory size |
|---|---|---|
| wolfSSL 4.7.0_6 (separate server WOLFSSL_CTX) | 5:58 | 987M |
| wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX) | 4:21 | 558M |
The Privoxy ports have been updated and now contain a wolfSSL-specific patch that prevents wolfSSL_shutdown() from being called at all to see if that works around the hangs under load (it doesn't).
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9970 hits Availability: 99.70 % Elapsed time: 596.25 secs Data transferred: 3.44 MB Response time: 36.24 secs Transaction rate: 16.72 trans/sec Throughput: 0.01 MB/sec Concurrency: 605.96 Successful transactions: 9487 Failed transactions: 30 Longest transaction: 463.42 Shortest transaction: 0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 50000 | grep -10 '^Detected inactivity' 2021-03-23 12:52:54.219 811897100 Header: scan: Content-Type: text/html 2021-03-23 12:52:54.219 811897100 Header: scan: Date: Tue, 23 Mar 2021 12:00:49 GMT 2021-03-23 12:52:54.219 811897100 Header: scan: ETag: W/"5f429d52-1fe" 2021-03-23 12:52:54.219 811897100 Header: scan: HTTP/1.1 200 OK 2021-03-23 12:52:54.219 811897100 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-23 12:52:54.219 811897100 Header: scan: Server: nginx 2021-03-23 12:52:54.219 811897100 Header: scan: Transfer-Encoding: chunked 2021-03-23 12:52:54.220 811897100 Connect: Closing client socket 73. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-23 12:52:54.220 811897100 Connect: Closing server socket 300 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5. 2021-03-23 12:52:54.220 811897100 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352. Detected inactivity: 148298 msecs 2021-03-23 12:55:22.518 81abdd700 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-23 12:55:22.519 817d88500 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-23 12:55:22.519 818d88b00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-23 12:55:22.519 81abdd700 Connect: Closing server socket 1168 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5. 2021-03-23 12:55:22.519 81abdd700 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/ 2021-03-23 12:55:22.520 817d88500 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/ 2021-03-23 12:55:22.520 818d88b00 Connect: Closing client socket 1055. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-23 12:55:22.520 818d88b00 Connect: Closing server socket 1148 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5. 2021-03-23 12:55:22.520 818d88b00 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/ 2021-03-23 12:55:22.520 81abdd700 Connect: Closing client socket 1118. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
Apparently it took nearly six minutes for the TLS handshake to fail but it also took more than 30 seconds for the Privoxy thread to call wolfSSL_connect() after the TCP connection was established:
[fk@privoxy-vm ~]$ grep 81abdd700 /var/log/privoxy/privoxy.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 5000 | grep -10 '^Detected inactivity' 2021-03-23 12:48:57.373 81abdd700 Header: scan: Accept-Encoding: gzip, deflate 2021-03-23 12:48:57.373 81abdd700 Header: scan: User-Agent: Mozilla/5.0 (portbld-amd64-freebsd11.4) Siege/4.0.7 2021-03-23 12:48:57.373 81abdd700 Header: scan: Connection: close 2021-03-23 12:48:57.373 81abdd700 Header: Keeping the client header 'Connection: close' around. The connection will not be kept alive. 2021-03-23 12:48:57.373 81abdd700 Header: Encrypted request processed 2021-03-23 12:48:57.373 81abdd700 Request: https://www.electrobsd.org/ 2021-03-23 12:48:57.373 81abdd700 Header: New HTTP Request-Line: GET / HTTP/1.1 2021-03-23 12:48:57.373 81abdd700 Connect: to www.electrobsd.org 2021-03-23 12:48:57.403 81abdd700 Connect: Connected to www.electrobsd.org[95.211.138.51]:443. 2021-03-23 12:48:57.461 81abdd700 Connect: Created new connection to www.electrobsd.org:443 on socket 1168. Detected inactivity: 34221 msecs 2021-03-23 12:49:31.682 81abdd700 Connect: Performing the TLS/SSL handshake with the server Detected inactivity: 350836 msecs 2021-03-23 12:55:22.518 81abdd700 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-23 12:55:22.519 81abdd700 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/ 2021-03-23 12:55:22.519 81abdd700 Connect: Closing server socket 1168 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5. 2021-03-23 12:55:22.520 81abdd700 Connect: Closing client socket 1118. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1.
| TLS library | CPU time | Memory size |
|---|---|---|
| wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX) | 4:37 | 573M |
The Privoxy ports have been updated. Worth mentioning is that privoxy-experimental-wolfssl calls wolfSSL_shutdown() again but now checks to see if the server socket is still alive before attempting to perform a TLS handshake.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9957 hits Availability: 99.57 % Elapsed time: 869.38 secs Data transferred: 3.50 MB Response time: 52.94 secs Transaction rate: 11.45 trans/sec Throughput: 0.00 MB/sec Concurrency: 606.30 Successful transactions: 9405 Failed transactions: 43 Longest transaction: 386.99 Shortest transaction: 0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324-wolfssl-c1000.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 50000 | grep -10 '^Detected inactivity' 2021-03-24 09:23:50.350 81cc24300 Header: scan: ETag: W/"5f429d52-1fe" 2021-03-24 09:23:50.350 81cc24300 Header: scan: HTTP/1.1 200 OK 2021-03-24 09:23:50.350 81cc24300 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-24 09:23:50.350 81cc24300 Header: scan: Server: nginx 2021-03-24 09:23:50.350 81cc24300 Header: scan: Transfer-Encoding: chunked 2021-03-24 09:23:50.351 81cc24300 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-24 09:23:50.351 81cc24300 Connect: Closing server socket 6 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5. 2021-03-24 09:23:50.351 81cc24300 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352. 2021-03-24 09:23:50.351 81cc24300 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2 2021-03-24 09:23:50.351 81cc24300 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2 Detected inactivity: 124595 msecs 2021-03-24 09:25:54.946 811d37d00 Connect: Failed to shutdown client connection on socket 1507. Attempts so far: 1, ret: 2 2021-03-24 09:25:54.946 811d37d00 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/ 2021-03-24 09:25:54.946 811d37d00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-24 09:25:54.946 811d3a500 Connect: Failed to shutdown client connection on socket 1511. Attempts so far: 1, ret: 2 2021-03-24 09:25:54.946 811d3a500 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/ 2021-03-24 09:25:54.946 811d3a500 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-24 09:25:54.946 811d3c300 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-24 09:25:54.947 811d37d00 Connect: Closing client socket 1507. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-24 09:25:54.947 811d37d00 Connect: Closing server socket 1476 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5. 2021-03-24 09:25:54.947 811d3c300 Connect: Failed to shutdown client connection on socket 1514. Attempts so far: 1, ret: 2 -- 2021-03-24 09:28:18.622 80fafef00 Header: scan: HTTP/1.1 200 OK 2021-03-24 09:28:18.622 80fafef00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-24 09:28:18.622 80fafef00 Header: scan: Server: nginx 2021-03-24 09:28:18.622 80fafef00 Header: scan: Transfer-Encoding: chunked 2021-03-24 09:28:18.623 80fafef00 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-24 09:28:18.623 80fafef00 Connect: Closing server socket 6 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5. 2021-03-24 09:28:18.623 80fafef00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352. 2021-03-24 09:28:18.623 80fafef00 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2 2021-03-24 09:28:18.623 80fafef00 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2 2021-03-24 09:28:18.623 80fafef00 Connect: Looks like we got the last chunk together with the server headers. We better stop reading. Detected inactivity: 52244 msecs 2021-03-24 09:29:10.867 817106900 Connect: Closing client socket 373. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-24 09:29:10.867 817106900 Connect: Failed to shutdown client connection on socket 373. Attempts so far: 1, ret: 2 2021-03-24 09:29:10.867 817106900 Connect: Failed to shutdown server connection on socket 441. Attempts so far: 2, ret: 2 2021-03-24 09:29:10.867 817106900 Connect: Not shutting down server connection on socket 441. The socket is no longer alive.
[fk@privoxy-vm ~]$ grep 'Skipping TLS handshake attempt' /var/log/privoxy/privoxy-3.0.33.20210324-wolfssl-c1000.log
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9958 hits Availability: 99.58 % Elapsed time: 885.62 secs Data transferred: 4.08 MB Response time: 37.20 secs Transaction rate: 11.24 trans/sec Throughput: 0.00 MB/sec Concurrency: 418.28 Successful transactions: 9368 Failed transactions: 42 Longest transaction: 445.58 Shortest transaction: 0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324-wolfssl-c1000-test-2.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 50000 | grep '^Detected inactivity' Detected inactivity: 67384 msecs Detected inactivity: 78784 msecs Detected inactivity: 262694 msecs [fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324-wolfssl-c1000-test-2.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 150000 | grep -10 '^Detected inactivity' 2021-03-24 09:55:07.892 808b62f00 Header: scan: Content-Encoding: gzip 2021-03-24 09:55:07.892 808b62f00 Header: scan: Content-Type: text/html 2021-03-24 09:55:07.892 808b62f00 Header: scan: Date: Wed, 24 Mar 2021 09:03:06 GMT 2021-03-24 09:55:07.892 808b62f00 Header: scan: ETag: W/"5f429d52-1fe" 2021-03-24 09:55:07.892 808b62f00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-24 09:55:07.892 808b62f00 Header: scan: Server: nginx 2021-03-24 09:55:07.892 808b62f00 Header: scan: Transfer-Encoding: chunked 2021-03-24 09:55:07.893 808b62f00 Connect: Closing client socket 7. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-24 09:55:07.893 808b62f00 Connect: Failed to shutdown client connection on socket 7. Attempts so far: 1, ret: 2 2021-03-24 09:55:07.893 808b62f00 Connect: Failed to shutdown server connection on socket 5. Attempts so far: 1, ret: 2 Detected inactivity: 262694 msecs 2021-03-24 09:59:30.587 816d74d00 Connect: Failed to shutdown server connection on socket 21. Attempts so far: 2, ret: 2 2021-03-24 09:59:30.587 816d74d00 Connect: Not shutting down server connection on socket 21. The socket is no longer alive. 2021-03-24 09:59:30.588 816d74d00 Connect: Closing client socket 24. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-24 09:59:30.588 816d74d00 Connect: Failed to shutdown client connection on socket 24. Attempts so far: 1, ret: 2 2021-03-24 09:59:30.591 802416000 Connect: Waiting for the next client connection. Currently active threads: 1 2021-03-24 09:59:30.591 828080700 Connect: Accepted connection from 172.16.1.6 on socket 5 2021-03-24 09:59:30.592 828080700 Header: addh-unique: Host: www.electrobsd.org:443 2021-03-24 09:59:30.592 828080700 Header: scan: CONNECT www.electrobsd.org:443 HTTP/1.0 2021-03-24 09:59:30.592 828080700 Header: scan: User-agent: Proxy-User 2021-03-24 09:59:30.593 828080700 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b
[fk@privoxy-vm ~]$ grep 'Skipping TLS handshake attempt' /var/log/privoxy/privoxy-3.0.33.20210324-wolfssl-c1000-test-2.log
| TLS library | CPU time | Memory size |
|---|---|---|
| wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX) 1 | 7:20 | 614M |
| wolfSSL 4.7.0_6 (shared server WOLFSSL_CTX) 2 | 5:08 | 577M |
The Privoxy ports have been updated, the wolfssl-specific code now has an option to control whether or not wolfSSL_shutdown() is called.
The wolfssl port has been recompiled with an updated patch to disable 3DES ciphers independently from 3DES support. The change is not expected to affect performance.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9967 hits Availability: 99.67 % Elapsed time: 737.46 secs Data transferred: 3.44 MB Response time: 18.42 secs Transaction rate: 13.52 trans/sec Throughput: 0.00 MB/sec Concurrency: 248.92 Successful transactions: 9528 Failed transactions: 33 Longest transaction: 262.83 Shortest transaction: 0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_1-wolfssl-tls-shutdown-enabled-c1000.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 50000 | grep '^Detected inactivity' Detected inactivity: 55803 msecs Detected inactivity: 54447 msecs Detected inactivity: 88680 msecs
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_1-wolfssl-tls-shutdown-enabled-c1000.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 80000 | grep -10 '^Detected inactivity' 2021-03-24 11:21:38.626 82e354b00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352. 2021-03-24 11:21:38.626 82e354b00 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2 2021-03-24 11:21:38.626 82e354b00 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2 2021-03-24 11:21:38.626 82e354b00 Connect: Looks like we got the last chunk together with the server headers. We better stop reading. 2021-03-24 11:21:38.626 82e354b00 Header: scan: Connection: close 2021-03-24 11:21:38.626 82e354b00 Header: scan: Content-Encoding: gzip 2021-03-24 11:21:38.626 82e354b00 Header: scan: Content-Type: text/html 2021-03-24 11:21:38.626 82e354b00 Header: scan: ETag: W/"5f429d52-1fe" 2021-03-24 11:21:38.626 82e354b00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-24 11:21:38.626 82e354b00 Header: scan: Transfer-Encoding: chunked Detected inactivity: 88680 msecs 2021-03-24 11:23:07.306 8196a0800 Connect: Failed to shutdown server connection on socket 18. Attempts so far: 2, ret: 2 2021-03-24 11:23:07.307 8196a0800 Connect: Closing client socket 16. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-24 11:23:07.307 8196a0800 Connect: Failed to shutdown client connection on socket 16. Attempts so far: 1, ret: 2 2021-03-24 11:23:07.307 8196a0800 Connect: Not shutting down server connection on socket 18. The socket is no longer alive. 2021-03-24 11:23:07.309 802416000 Connect: Waiting for the next client connection. Currently active threads: 1 2021-03-24 11:23:07.309 82455a100 Connect: Accepted connection from 172.16.1.6 on socket 5 2021-03-24 11:23:07.309 82455a100 Header: addh-unique: Host: www.electrobsd.org:443 2021-03-24 11:23:07.309 82455a100 Header: scan: CONNECT www.electrobsd.org:443 HTTP/1.0 2021-03-24 11:23:07.309 82455a100 Header: scan: User-agent: Proxy-User 2021-03-24 11:23:07.311 82455a100 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 10000 hits Availability: 100.00 % Elapsed time: 590.26 secs Data transferred: 3.23 MB Response time: 28.22 secs Transaction rate: 16.94 trans/sec Throughput: 0.01 MB/sec Concurrency: 478.16 Successful transactions: 9601 Failed transactions: 0 Longest transaction: 400.39 Shortest transaction: 0.07
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_1-wolfssl-tls-shutdown-disabled-c1000.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep '^Detected inactivity' Detected inactivity: 12664 msecs Detected inactivity: 205247 msecs Detected inactivity: 65135 msecs [fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_1-wolfssl-tls-shutdown-disabled-c1000.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 100000 | grep -10 '^Detected inactivity' 2021-03-24 11:33:10.470 82a4cde00 Header: scan: Date: Wed, 24 Mar 2021 10:41:09 GMT 2021-03-24 11:33:10.470 82a4cde00 Header: scan: ETag: W/"5f429d52-1fe" 2021-03-24 11:33:10.470 82a4cde00 Header: scan: HTTP/1.1 200 OK 2021-03-24 11:33:10.470 82a4cde00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-24 11:33:10.470 82a4cde00 Header: scan: Server: nginx 2021-03-24 11:33:10.470 82a4cde00 Header: scan: Transfer-Encoding: chunked 2021-03-24 11:33:10.471 82a4cde00 Connect: Closing client socket 7. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-24 11:33:10.471 82a4cde00 Connect: Closing server socket 8 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5. 2021-03-24 11:33:10.471 82a4cde00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352. 2021-03-24 11:33:10.473 82a4cde00 Connect: Drained 31 bytes before closing socket 7 Detected inactivity: 205247 msecs 2021-03-24 11:36:35.720 817198400 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-24 11:36:35.721 817198400 Connect: Closing client socket 966. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-24 11:36:35.721 817198400 Connect: Closing server socket 1039 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 0. Timeout: 5. 2021-03-24 11:36:35.721 817198400 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/ 2021-03-24 11:36:35.726 802416000 Connect: Waiting for the next client connection. Currently active threads: 399 2021-03-24 11:36:35.726 82552cd00 Connect: Accepted connection from 172.16.1.6 on socket 5 2021-03-24 11:36:35.726 82552cd00 Header: scan: CONNECT www.electrobsd.org:443 HTTP/1.0 2021-03-24 11:36:35.726 82552cd00 Header: scan: User-agent: Proxy-User 2021-03-24 11:36:35.727 82552cd00 Header: addh-unique: Host: www.electrobsd.org:443 2021-03-24 11:36:35.728 82552cd00 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b
| TLS library | CPU time | Memory size |
|---|---|---|
| wolfSSL 4.7.0_7 (shared server WOLFSSL_CTX, shutdown-tls-connections 1) | 2:52 | 569M |
| wolfSSL 4.7.0_7 (shared server WOLFSSL_CTX, shutdown-tls-connections 0) | 3:34 | 524M |
Apparently not explicitly calling wolfSSL_shutdown() actually increased the cpu time ...
The privoxy-experimental-wolfssl and wolfssl ports have been recompiled with option DEBUG enabled. This is obviously expected to affect performance but may help to diagnose the hangs.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9971 hits Availability: 99.71 % Elapsed time: 791.51 secs Data transferred: 3.41 MB Response time: 16.97 secs Transaction rate: 12.60 trans/sec Throughput: 0.00 MB/sec Concurrency: 213.72 Successful transactions: 9489 Failed transactions: 29 Longest transaction: 633.22 Shortest transaction: 0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_2-wolfssl-tls-shutdown-enabled-c1000.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep '^Detected inactivity' Detected inactivity: 17208 msecs Detected inactivity: 12763 msecs Detected inactivity: 28067 msecs Detected inactivity: 45745 msecs Detected inactivity: 20359 msecs Detected inactivity: 10323 msecs Detected inactivity: 14201 msecs Detected inactivity: 15173 msecs Detected inactivity: 15198 msecs Detected inactivity: 26831 msecs Detected inactivity: 25923 msecs Detected inactivity: 62608 msecs [fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_2-wolfssl-tls-shutdown-enabled-c1000.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 50000 | grep -10 '^Detected inactivity' 2021-03-24 12:35:23.209 819524b00 Header: scan: ETag: W/"5f429d52-1fe" 2021-03-24 12:35:23.209 819524b00 Header: scan: HTTP/1.1 200 OK 2021-03-24 12:35:23.209 819524b00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-24 12:35:23.209 819524b00 Header: scan: Server: nginx 2021-03-24 12:35:23.209 819524b00 Header: scan: Transfer-Encoding: chunked 2021-03-24 12:35:23.210 819524b00 Connect: Closing client socket 7. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-24 12:35:23.210 819524b00 Connect: Closing server socket 8 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5. 2021-03-24 12:35:23.210 819524b00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352. 2021-03-24 12:35:23.210 819524b00 Connect: Failed to shutdown client connection on socket 7. Attempts so far: 1, ret: 2 2021-03-24 12:35:23.210 819524b00 Connect: Failed to shutdown server connection on socket 8. Attempts so far: 1, ret: 2 Detected inactivity: 62608 msecs 2021-03-24 12:36:25.818 80cc31b00 Connect: Closing client socket 51. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-24 12:36:25.818 80cc31b00 Connect: Failed to shutdown client connection on socket 51. Attempts so far: 1, ret: 2 2021-03-24 12:36:25.818 80cc31b00 Connect: Failed to shutdown server connection on socket 166. Attempts so far: 2, ret: 2 2021-03-24 12:36:25.818 80cc31b00 Connect: Not shutting down server connection on socket 166. The socket is no longer alive. 2021-03-24 12:36:25.821 802416000 Connect: Waiting for the next client connection. Currently active threads: 1 2021-03-24 12:36:25.821 8235c3500 Connect: Accepted connection from 172.16.1.6 on socket 5 2021-03-24 12:36:25.822 8235c3500 Header: addh-unique: Host: www.electrobsd.org:443 2021-03-24 12:36:25.822 8235c3500 Header: scan: CONNECT www.electrobsd.org:443 HTTP/1.0 2021-03-24 12:36:25.822 8235c3500 Header: scan: User-agent: Proxy-User 2021-03-24 12:36:25.824 8235c3500 Connect: Performing the TLS/SSL handshake with client. Hash of host: 6db5da8a16c246d1bd8c0fa7cd160a5b
| TLS library | CPU time | Memory size |
|---|---|---|
| wolfSSL 4.7.0_7 (shared server WOLFSSL_CTX, shutdown-tls-connections 1) | 3:20 | 586M |
According to gdb the idling Privoxy had lots of threads blocking like this:
Thread 2 (LWP 100598 of process 83579): #0 0x000000080197297a in _recvfrom () from /lib/libc.so.7 #1 0x000000080165ecbf in __thr_recvfrom (s=812, b=0x8249b6470, l=5, f=0, from=0x0, fl=0x0) at /usr/src/lib/libthr/thread/thr_syscalls.c:456 #2 0x000000080122c698 in wolfIO_Recv (sd=812, buf=0x8249b6470 "", sz=5, rdFlags=0) at src/wolfio.c:700 #3 0x000000080122c569 in EmbedReceive (ssl=0x8249b6200, buf=0x8249b6470 "", sz=5, ctx=0x8249bf7f4) at src/wolfio.c:233 #4 0x00000008012252d5 in wolfSSLReceive (ssl=0x8249b6200, buf=0x8249b6470 "", sz=5) at src/internal.c:8098 #5 0x0000000801212646 in GetInputData (ssl=0x8249b6200, size=5) at src/internal.c:15090 #6 0x0000000801210d9e in ProcessReply (ssl=0x8249b6200) at src/internal.c:15286 #7 0x0000000801242b93 in wolfSSL_connect (ssl=0x8249b6200) at src/ssl.c:12504 #8 0x0000000000436b5f in ?? () #9 0x0000000000423c42 in ?? () #10 0x000000080165c08c in thread_start (curthread=0x80a1a9300) at /usr/src/lib/libthr/thread/thr_create.c:290 #11 0x0000000000000000 in ?? () Backtrace stopped: Cannot access memory at address 0x7fff8d369000
It also turned out that building the port Privoxy with the DEBUG option enabled did not disable stripping ...
The privoxy-experimental-wolfssl port has been updated and now contains a patch to optionally use a non-default receive callback with a timeout. The DEBUG option is still being used and stripping has been disabled properly (I hope).
Privoxy was configured to shutdown TLS connections and to use a receive callback timeout of 10 seconds.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9934 hits Availability: 99.34 % Elapsed time: 468.43 secs Data transferred: 3.63 MB Response time: 31.75 secs Transaction rate: 21.21 trans/sec Throughput: 0.01 MB/sec Concurrency: 673.42 Successful transactions: 8958 Failed transactions: 66 Longest transaction: 89.88 Shortest transaction: 0.07
The Longest transaction
and the Elapsed time
went both down.
I love it when a plan comes together.
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_4-wolfssl-tls-callback-timeout-10-c1000.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -2 '^Detected inactivity' 2021-03-24 15:48:51.240 814505300 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 15:48:52.467 817210c00 Connect: Connected to www.electrobsd.org[95.211.138.51]:443. Detected inactivity: 12120 msecs 2021-03-24 15:49:04.587 80c6d0300 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 15:49:04.591 80b656500 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). -- 2021-03-24 15:51:20.017 80c40ee00 Connect: Connected to www.electrobsd.org[95.211.138.51]:443. 2021-03-24 15:51:20.076 814344e00 Connect: Connected to www.electrobsd.org[95.211.138.51]:443. Detected inactivity: 17795 msecs 2021-03-24 15:51:37.871 8150c6b00 Error: Gave up waiting for TLS data. Timeout: 10 seconds. 2021-03-24 15:51:37.871 81cee5000 Error: Gave up waiting for TLS data. Timeout: 10 seconds. -- 2021-03-24 15:53:07.036 813f5d200 Connect: Failed to shutdown server connection on socket 194. Attempts so far: 6, ret: 2 2021-03-24 15:53:07.036 813f5d200 Error: Gave up waiting for TLS data. Timeout: 10 seconds. Detected inactivity: 10066 msecs 2021-03-24 15:53:17.102 813f5d200 Connect: Failed to shutdown server connection on socket 194. Attempts so far: 7, ret: 2 2021-03-24 15:53:17.102 813f5d200 Error: Gave up waiting for TLS data. Timeout: 10 seconds. Detected inactivity: 10034 msecs 2021-03-24 15:53:27.136 813f5d200 Connect: Failed to shutdown server connection on socket 194. Attempts so far: 8, ret: 2 2021-03-24 15:53:27.136 813f5d200 Error: Gave up waiting for TLS data. Timeout: 10 seconds. Detected inactivity: 10030 msecs 2021-03-24 15:53:37.166 813f5d200 Connect: Failed to shutdown server connection on socket 194. Attempts so far: 9, ret: 2 2021-03-24 15:53:37.166 813f5d200 Error: Gave up waiting for TLS data. Timeout: 10 seconds. Detected inactivity: 10012 msecs 2021-03-24 15:53:47.178 813f5d200 Connect: Closing client socket 23. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-24 15:53:47.178 813f5d200 Connect: Failed to shutdown client connection on socket 23. Attempts so far: 1, ret: 2
[fk@privoxy-vm ~]$ grep -c 'Gave up waiting for TLS data' /var/log/privoxy/privoxy-3.0.33.20210324_4-wolfssl-tls-callback-timeout-10-c1000.log 994
| TLS library | CPU time | Memory size |
|---|---|---|
| wolfSSL 4.7.0_7 (shared server WOLFSSL_CTX, shutdown-tls-connections 1, tls-receive-callback-timeout 10) | 5:58 | 556M |
The Privoxy ports have been updated. The privoxy-experimental-wolfssl port is no longer build with the DEBUG option enabled and the receive callback patch has been slightly refined.
The wolfssl port has been recompiled with the DEBUG option disabled as well.
Privoxy has been configured with:
reuse-server-tls-context 1 shutdown-tls-connections 1 tls-receive-callback-timeout 20
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9700 hits Availability: 97.00 % Elapsed time: 675.19 secs Data transferred: 7.23 MB Response time: 40.55 secs Transaction rate: 14.37 trans/sec Throughput: 0.01 MB/sec Concurrency: 582.57 Successful transactions: 8224 Failed transactions: 300 Longest transaction: 149.05 Shortest transaction: 0.06
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-10-1.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 15000 | grep -10 '^Detected inactivity' 2021-03-24 17:02:47.049 819cbba00 Header: scan: HTTP/1.1 200 OK 2021-03-24 17:02:47.049 819cbba00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-24 17:02:47.049 819cbba00 Header: scan: Server: nginx 2021-03-24 17:02:47.049 819cbba00 Header: scan: Transfer-Encoding: chunked 2021-03-24 17:02:47.050 819cbba00 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-24 17:02:47.050 819cbba00 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2 2021-03-24 17:02:51.656 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 5, ret: 2 2021-03-24 17:02:51.656 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:02:53.656 8061aeb00 Connect: Failed to shutdown server connection on socket 529. Attempts so far: 7, ret: 2 2021-03-24 17:02:53.656 8061aeb00 Error: Gave up waiting for TLS data on socket 529. Timeout: 20 seconds. Detected inactivity: 18012 msecs 2021-03-24 17:03:11.668 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:03:11.669 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 6, ret: 2 2021-03-24 17:03:13.758 8061aeb00 Connect: Failed to shutdown server connection on socket 529. Attempts so far: 8, ret: 2 2021-03-24 17:03:13.758 8061aeb00 Error: Gave up waiting for TLS data on socket 529. Timeout: 20 seconds. Detected inactivity: 17947 msecs 2021-03-24 17:03:31.705 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:03:31.706 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 7, ret: 2 2021-03-24 17:03:33.855 8061aeb00 Connect: Failed to shutdown server connection on socket 529. Attempts so far: 9, ret: 2 2021-03-24 17:03:33.855 8061aeb00 Error: Gave up waiting for TLS data on socket 529. Timeout: 20 seconds. Detected inactivity: 17871 msecs 2021-03-24 17:03:51.726 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 8, ret: 2 2021-03-24 17:03:51.726 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:03:53.877 8061aeb00 Connect: Closing client socket 355. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-24 17:03:53.877 8061aeb00 Connect: Failed to shutdown server connection on socket 529. Attempts so far: 10, ret: 2 2021-03-24 17:03:53.877 8061aeb00 Error: Failed to shutdown server connection on socket 529 after 10 attempts. ret: 2, error: 0, unknown error number 2021-03-24 17:03:53.877 8061aeb00 Error: Gave up waiting for TLS data on socket 529. Timeout: 20 seconds. 2021-03-24 17:03:53.878 8061aeb00 Connect: Failed to shutdown client connection on socket 355. Attempts so far: 1, ret: 2 2021-03-24 17:03:53.880 802416000 Connect: Waiting for the next client connection. Currently active threads: 2 2021-03-24 17:03:53.880 80a6a2600 Connect: Accepted connection from 172.16.1.6 on socket 5 2021-03-24 17:03:53.880 80a6a2600 Header: addh-unique: Host: www.electrobsd.org:443 -- 2021-03-24 17:03:53.974 80a6a2600 Connect: Looks like we got the last chunk together with the server headers. We better stop reading. 2021-03-24 17:03:53.974 80a6a2600 Header: scan: Connection: close 2021-03-24 17:03:53.974 80a6a2600 Header: scan: Content-Encoding: gzip 2021-03-24 17:03:53.974 80a6a2600 Header: scan: Content-Type: text/html 2021-03-24 17:03:53.974 80a6a2600 Header: scan: Date: Wed, 24 Mar 2021 16:11:53 GMT 2021-03-24 17:03:53.974 80a6a2600 Header: scan: ETag: W/"5f429d52-1fe" 2021-03-24 17:03:53.974 80a6a2600 Header: scan: HTTP/1.1 200 OK 2021-03-24 17:03:53.974 80a6a2600 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-24 17:03:53.974 80a6a2600 Header: scan: Server: nginx 2021-03-24 17:03:53.974 80a6a2600 Header: scan: Transfer-Encoding: chunked Detected inactivity: 17814 msecs 2021-03-24 17:04:11.788 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 9, ret: 2 2021-03-24 17:04:11.788 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. Detected inactivity: 20033 msecs 2021-03-24 17:04:31.821 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:04:31.822 81562cc00 Connect: Closing client socket 293. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-24 17:04:31.822 81562cc00 Connect: Failed to shutdown client connection on socket 293. Attempts so far: 1, ret: 2 2021-03-24 17:04:31.822 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 10, ret: 2 2021-03-24 17:04:31.822 81562cc00 Error: Failed to shutdown server connection on socket 758 after 10 attempts. ret: 2, error: 0, unknown error number
[fk@privoxy-vm ~]$ grep 81562cc00 /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-10-1.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 15000 | grep -10 '^Detected' 2021-03-24 17:01:31.519 81562cc00 Header: scan: Content-Type: text/html 2021-03-24 17:01:31.519 81562cc00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-24 17:01:31.519 81562cc00 Header: scan: Transfer-Encoding: chunked 2021-03-24 17:01:31.519 81562cc00 Header: scan: Connection: close 2021-03-24 17:01:31.519 81562cc00 Header: scan: ETag: W/"5f429d52-1fe" 2021-03-24 17:01:31.519 81562cc00 Header: scan: Content-Encoding: gzip 2021-03-24 17:01:31.519 81562cc00 Connect: Looks like we got the last chunk together with the server headers. We better stop reading. 2021-03-24 17:01:31.519 81562cc00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352. 2021-03-24 17:01:31.519 81562cc00 Connect: Closing server socket 758 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5. 2021-03-24 17:01:31.519 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 1, ret: 2 Detected inactivity: 20017 msecs 2021-03-24 17:01:51.536 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:01:51.536 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 2, ret: 2 Detected inactivity: 20020 msecs 2021-03-24 17:02:11.556 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:02:11.556 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 3, ret: 2 Detected inactivity: 20080 msecs 2021-03-24 17:02:31.636 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:02:31.636 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 4, ret: 2 Detected inactivity: 20020 msecs 2021-03-24 17:02:51.656 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:02:51.656 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 5, ret: 2 Detected inactivity: 20012 msecs 2021-03-24 17:03:11.668 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:03:11.669 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 6, ret: 2 Detected inactivity: 20036 msecs 2021-03-24 17:03:31.705 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:03:31.706 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 7, ret: 2 Detected inactivity: 20020 msecs 2021-03-24 17:03:51.726 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:03:51.726 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 8, ret: 2 Detected inactivity: 20062 msecs 2021-03-24 17:04:11.788 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:04:11.788 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 9, ret: 2 Detected inactivity: 20033 msecs 2021-03-24 17:04:31.821 81562cc00 Error: Gave up waiting for TLS data on socket 758. Timeout: 20 seconds. 2021-03-24 17:04:31.822 81562cc00 Connect: Failed to shutdown server connection on socket 758. Attempts so far: 10, ret: 2 2021-03-24 17:04:31.822 81562cc00 Error: Failed to shutdown server connection on socket 758 after 10 attempts. ret: 2, error: 0, unknown error number 2021-03-24 17:04:31.822 81562cc00 Connect: Closing client socket 293. Keep-alive: 0. Socket alive: 1. Data available: 1. Configuration file change detected: 0. Requests received: 1. 2021-03-24 17:04:31.822 81562cc00 Connect: Failed to shutdown client connection on socket 293. Attempts so far: 1, ret: 2
Looks like the callback is getting called again even if it already returned a timeout for the socket.
I've changed shutdown-tls-connections to 0 and repeated the test.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9991 hits Availability: 99.91 % Elapsed time: 512.08 secs Data transferred: 3.81 MB Response time: 39.53 secs Transaction rate: 19.51 trans/sec Throughput: 0.01 MB/sec Concurrency: 771.23 Successful transactions: 8116 Failed transactions: 9 Longest transaction: 126.15 Shortest transaction: 0.06
Looks like the Availability
went up quite a bit.
More requests were served successfully in less time.
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-10-no-shutdown-2.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 10000 | grep -10 '^Detected inactivity' 2021-03-24 17:13:31.059 803037700 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:13:31.059 803219500 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:13:31.059 80331b400 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:13:31.059 8037c4b00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:13:31.059 80872f100 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:13:31.059 80b8f4700 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:13:31.788 803b51b00 Header: scan: Transfer-Encoding: chunked 2021-03-24 17:13:33.218 803b51b00 Header: scan: Connection: close 2021-03-24 17:13:34.731 803b51b00 Header: scan: ETag: W/"5f429d52-1fe" 2021-03-24 17:13:37.068 803b51b00 Header: scan: Content-Encoding: gzip Detected inactivity: 10482 msecs 2021-03-24 17:13:47.550 803b51b00 Connect: Looks like we got the last chunk together with the server headers. We better stop reading. 2021-03-24 17:13:56.409 803b51b00 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352. 2021-03-24 17:13:56.409 804388700 Header: scan: Server: nginx 2021-03-24 17:13:56.826 804388700 Header: scan: Date: Wed, 24 Mar 2021 16:21:25 GMT 2021-03-24 17:13:56.961 804388700 Connect: Closing server socket 113 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 0. Socket alive: 1. Timeout: 5. 2021-03-24 17:13:56.961 804388700 Connect: Done reading from server. Content length: 352 as expected. Bytes most recently read: 352. 2021-03-24 17:13:56.961 804388700 Connect: Looks like we got the last chunk together with the server headers. We better stop reading. 2021-03-24 17:13:56.961 804388700 Header: scan: Connection: close 2021-03-24 17:13:56.961 804388700 Header: scan: Content-Encoding: gzip 2021-03-24 17:13:56.961 804388700 Header: scan: Content-Type: text/html
I've changed shutdown-tls-connections back to 1 and reduced tls-receive-callback-timeout to 5.
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out Transactions: 9986 hits Availability: 99.86 % Elapsed time: 215.26 secs Data transferred: 3.64 MB Response time: 14.02 secs Transaction rate: 46.39 trans/sec Throughput: 0.02 MB/sec Concurrency: 650.44 Successful transactions: 1354 Failed transactions: 14 Longest transaction: 65.70 Shortest transaction: 0.06
[fk@privoxy-vm ~]$ grep -c 'Gave up waiting for TLS data' /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-5-shutdown-1.log 8632
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-5-shutdown-1.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 3000 | grep -5 '^Detected inactivity' 2021-03-24 17:57:44.446 806ce9d00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:57:44.447 80411f900 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:57:44.448 804057d00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:57:44.521 804058700 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:57:45.017 8025e5600 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). Detected inactivity: 4944 msecs 2021-03-24 17:57:49.961 808991b00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:57:49.962 802975000 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:57:49.962 804120300 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:57:49.962 806899b00 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). 2021-03-24 17:57:49.963 802667700 Connect: Server successfully connected over TLSv1.2 (ECDHE-RSA-AES128-GCM-SHA256). -- 2021-03-24 17:57:58.667 818860f00 Header: scan: Last-Modified: Sun, 23 Aug 2020 16:46:10 GMT 2021-03-24 17:57:58.667 818860f00 Header: scan: Server: nginx 2021-03-24 17:57:58.667 818860f00 Header: scan: Transfer-Encoding: chunked 2021-03-24 17:57:58.668 818860f00 Connect: Failed to shutdown server connection on socket 1062. Attempts so far: 1, ret: 2 2021-03-24 17:57:59.228 807c31400 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket Detected inactivity: 3082 msecs 2021-03-24 17:58:02.310 8063d5c00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-24 17:58:02.402 806c76900 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-24 17:58:02.958 806c75f00 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-24 17:58:04.068 807c33700 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-24 17:58:06.788 8089a7000 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket -- 2021-03-24 18:00:19.967 80c0c9c00 Header: scan: Server: nginx 2021-03-24 18:00:19.967 80c0c9c00 Header: scan: Transfer-Encoding: chunked 2021-03-24 18:00:19.968 80c0c9c00 Connect: Closing client socket 5. Keep-alive: 0. Socket alive: 1. Data available: 0. Configuration file change detected: 0. Requests received: 1. 2021-03-24 18:00:19.968 80c0c9c00 Connect: Failed to shutdown client connection on socket 5. Attempts so far: 1, ret: 2 2021-03-24 18:00:19.968 80c0c9c00 Connect: Failed to shutdown server connection on socket 6. Attempts so far: 1, ret: 2 Detected inactivity: 8027 msecs 2021-03-24 18:00:27.995 80c456800 Connect: Could not connect to [www.electrobsd.org]:443: Operation timed out. 2021-03-24 18:00:27.996 80c456800 Connect: Failed to shutdown client connection on socket 672. Attempts so far: 1, ret: 2 2021-03-24 18:00:27.996 80c456800 Crunch: Connection failure: https://www.electrobsd.org/ 2021-03-24 18:00:27.999 802416000 Connect: Waiting for the next client connection. Currently active threads: 7 2021-03-24 18:00:27.999 811b8f400 Connect: Accepted connection from 172.16.1.6 on socket 5
Most of the requests failed.
[fk@privoxy-vm ~]$ privoxy-log-parser.pl --statistics /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-5-shutdown-1.log
Client requests total: 9999
Crunches: 8645 (86.46%)
Blocks: 0 (0.00%)
Fast redirections: 0 (0.00%)
Connection timeouts: 0 (0.00%)
Connection failures: 13 (0.13%)
Outgoing requests: 1354 (13.54%)
Server keep-alive offers: 0 (0.00%)
New outgoing connections: 1354 (13.54%)
Reused server connections: 0 (0.00%; server offers accepted: 0.00%)
Empty responses: 0 (0.00%)
Empty responses on new connections: 0 (0.00%)
Empty responses on reused connections: 0 (0.00%)
Client connections: 9986
Bytes of content transferred to the client: 4078522
Client requests per connection distribution:
9986: 1
Enable --show-complete-request-distribution to get less common numbers as well.
Improperly accounted requests: ~13
Method distribution:
9999 : GET
Client HTTP versions:
9999 : HTTP/1.1
HTTP status codes according to 'debug 512' (status codes sent by the server may differ):
8632 : 403
1354 : 200
13 : 503
URL statistics are disabled. Increase --url-statistics-threshold to enable them.
Passed request statistics are disabled. Increase --passed-request-statistics-threshold to enable them.
Host statistics are disabled. Increase --host-statistics-threshold to enable them.
Privoxy has been configured with:
reuse-server-tls-context 0 shutdown-tls-connections 1 tls-receive-callback-timeout 20
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9986 hits Availability: 99.86 % Elapsed time: 446.16 secs Data transferred: 3.33 MB Response time: 33.71 secs Transaction rate: 22.38 trans/sec Throughput: 0.01 MB/sec Concurrency: 754.53 Successful transactions: 9889 Failed transactions: 14 Longest transaction: 76.36 Shortest transaction: 0.20
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210324_5_tls-timeout-20-no-server-ctx-sharing.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 5000 | grep -c '^Detected inactivity' 0
| TLS library | CPU time | Memory size |
|---|---|---|
| wolfSSL 4.7.0_8 (shared server WOLFSSL_CTX, shutdown-tls-connections 1, tls-receive-callback-timeout 20) | 7:14 | 625M |
| wolfSSL 4.7.0_8 (shared server WOLFSSL_CTX, shutdown-tls-connections 0, tls-receive-callback-timeout 20) | 7:12 | ??? |
| wolfSSL 4.7.0_8 (shared server WOLFSSL_CTX, shutdown-tls-connections 1, tls-receive-callback-timeout 5) | 2:35 | 441M |
| wolfSSL 4.7.0_8 (server WOLFSSL_CTX not shared, shutdown-tls-connections 1, tls-receive-callback-timeout 20) | 6:42 | 979M |
As it turned out there was a memory leak in case of "reuse-server-tls-context 0" ...
The Privoxy ports have been updated. The wolfssl-specific code has been changed to fix the memory leak in case of "reuse-server-tls-context 0" and the callback now logs a message after each second spent waiting for data (to rule out a thread scheduling issue).
Privoxy has been configured with:
reuse-server-tls-context 1 shutdown-tls-connections 1 tls-receive-callback-timeout 20
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9905 hits Availability: 99.05 % Elapsed time: 489.04 secs Data transferred: 3.86 MB Response time: 33.55 secs Transaction rate: 20.25 trans/sec Throughput: 0.01 MB/sec Concurrency: 679.60 Successful transactions: 9069 Failed transactions: 95 Longest transaction: 113.94 Shortest transaction: 0.08
[fk@privoxy-vm ~]$ sort /var/log/privoxy/privoxy-3.0.33.20210325_tls--no-server-ctx-sharing.log | privoxy-log-parser.pl --detect-inactivity --inactivity-threshold 5000 | grep '^Detected' Detected inactivity: 5499 msecs Detected inactivity: 5756 msecs Detected inactivity: 8873 msecs Detected inactivity: 5110 msecs Detected inactivity: 13424 msecs Detected inactivity: 7807 msecs
[fk@privoxy-vm ~]$ grep -c "Gave up" /var/log/privoxy/privoxy-3.0.33.20210325_tls--no-server-ctx-sharing.log 837
Looking at one of those failures:
[fk@privoxy-vm ~]$ grep 825b70600 /var/log/privoxy/privoxy-3.0.33.20210325_tls--no-server-ctx-sharing.log | grep -30 'Error: Gave up waiting for TLS' 2021-03-25 04:37:21.230 825b70600 Header: scan: Accept-Encoding: gzip, deflate 2021-03-25 04:37:21.230 825b70600 Header: scan: User-Agent: Mozilla/5.0 (portbld-amd64-freebsd11.4) Siege/4.0.7 2021-03-25 04:37:21.230 825b70600 Header: scan: Connection: close 2021-03-25 04:37:21.230 825b70600 Header: Keeping the client header 'Connection: close' around. The connection will not be kept alive. 2021-03-25 04:37:21.230 825b70600 Header: Encrypted request processed 2021-03-25 04:37:21.230 825b70600 Request: https://www.electrobsd.org/ 2021-03-25 04:37:21.230 825b70600 Header: New HTTP Request-Line: GET / HTTP/1.1 2021-03-25 04:37:21.230 825b70600 Connect: to www.electrobsd.org 2021-03-25 04:37:21.241 825b70600 Connect: Connected to www.electrobsd.org[95.211.138.51]:443. 2021-03-25 04:37:21.241 825b70600 Connect: Created new connection to www.electrobsd.org:443 on socket 1308. 2021-03-25 04:37:22.591 825b70600 Connect: Performing the TLS/SSL handshake with the server 2021-03-25 04:37:23.642 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 1 seconds. 2021-03-25 04:37:24.695 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 2 seconds. 2021-03-25 04:37:25.729 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 3 seconds. 2021-03-25 04:38:30.739 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 4 seconds. 2021-03-25 04:38:31.759 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 5 seconds. 2021-03-25 04:38:32.888 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 6 seconds. 2021-03-25 04:38:33.898 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 7 seconds. 2021-03-25 04:38:35.344 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 8 seconds. 2021-03-25 04:38:36.389 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 9 seconds. 2021-03-25 04:38:37.568 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 10 seconds. 2021-03-25 04:38:38.585 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 11 seconds. 2021-03-25 04:38:39.641 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 12 seconds. 2021-03-25 04:38:40.650 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 13 seconds. 2021-03-25 04:38:41.668 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 14 seconds. 2021-03-25 04:38:42.678 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 15 seconds. 2021-03-25 04:38:43.689 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 16 seconds. 2021-03-25 04:38:44.698 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 17 seconds. 2021-03-25 04:38:45.715 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 18 seconds. 2021-03-25 04:38:46.782 825b70600 Connect: Continuing to wait for TLS data on socket 1308. Time spent waiting so far: 19 seconds. 2021-03-25 04:38:47.908 825b70600 Error: Gave up waiting for TLS data on socket 1308. Timeout: 20 seconds. 2021-03-25 04:38:48.839 825b70600 Error: X509 certificate verification for www.electrobsd.org failed with error -308: error state on socket 2021-03-25 04:38:48.840 825b70600 Crunch: Certificate error: error state on socket: https://www.electrobsd.org/ 2021-03-25 04:38:48.840 825b70600 Connect: Failed to shutdown client connection on socket 1195. Attempts so far: 1, ret: 2 2021-03-25 04:38:49.440 825b70600 Connect: Closing server socket 1308 connected to www.electrobsd.org. Keep-alive: 0. Tainted: 1. Socket alive: 1. Timeout: 5. 2021-03-25 04:38:49.441 825b70600 Connect: Closing client socket 1195. Keep-alive: 0. Socket alive: 0. Data available: 0. Configuration file change detected: 0. Requests received: 1.
So the problem doesn't seem to be a scheduling issue.
Privoxy has been configured with:
reuse-server-tls-context 0 shutdown-tls-connections 1 tls-receive-callback-timeout 20
[fk@benchmark-vm ~]$ siege --concurrent=1000 --reps=10 --benchmark "https://www.electrobsd.org/" ** SIEGE 4.0.7 ** Preparing 1000 concurrent users for battle. The server is now under siege... Transactions: 9969 hits Availability: 99.69 % Elapsed time: 321.38 secs Data transferred: 3.43 MB Response time: 23.46 secs Transaction rate: 31.02 trans/sec Throughput: 0.01 MB/sec Concurrency: 727.84 Successful transactions: 9656 Failed transactions: 31 Longest transaction: 73.66 Shortest transaction: 0.08
[fk@privoxy-vm ~]$ grep -c "Gave up" /var/log/privoxy/privoxy-3.0.33.20210325_tls-server-ctx-shared.log 316
Apparently not sharing the server WOLFSSL_CTX results in better performance and requires less resident memory which seem non-intuitive.
| TLS library | CPU time | Memory size |
|---|---|---|
| wolfSSL 4.7.0_9 (reuse-server-tls-context 1) | 6:39 | 588M |
| wolfSSL 4.7.0_9 (reuse-server-tls-context 0) | 4:33 | 402M |
This test is fetching 1 MB of random data. I reduced the concurrency level to 10 and increased the reps to 100.
Note that the network connections of the server hosting www.electrobsd.org and the system hosting the VMs are fairly busy so Privoxy spends a fair amount of time waiting for data to arrive.
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege...[alert] socket: select and discovered it's not ready sock.c:384: Operation timed out [alert] socket: read check timed out(30) sock.c:273: Operation timed out Transactions: 999 hits Availability: 99.90 % Elapsed time: 217.02 secs Data transferred: 999.00 MB Response time: 2.07 secs Transaction rate: 4.60 trans/sec Throughput: 4.60 MB/sec Concurrency: 9.52 Successful transactions: 999 Failed transactions: 1 Longest transaction: 10.62 Shortest transaction: 0.33
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 999 hits Availability: 99.90 % Elapsed time: 224.62 secs Data transferred: 999.01 MB Response time: 2.13 secs Transaction rate: 4.45 trans/sec Throughput: 4.45 MB/sec Concurrency: 9.49 Successful transactions: 999 Failed transactions: 1 Longest transaction: 30.06 Shortest transaction: 0.34
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 195.53 secs Data transferred: 1000.00 MB Response time: 1.91 secs Transaction rate: 5.11 trans/sec Throughput: 5.11 MB/sec Concurrency: 9.77 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 6.92 Shortest transaction: 0.35
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 215.38 secs Data transferred: 1000.00 MB Response time: 2.11 secs Transaction rate: 4.64 trans/sec Throughput: 4.64 MB/sec Concurrency: 9.78 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 7.66 Shortest transaction: 0.40
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 267.44 secs Data transferred: 1000.00 MB Response time: 2.59 secs Transaction rate: 3.74 trans/sec Throughput: 3.74 MB/sec Concurrency: 9.69 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 9.52 Shortest transaction: 0.80
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 998 hits Availability: 99.80 % Elapsed time: 269.85 secs Data transferred: 998.01 MB Response time: 2.52 secs Transaction rate: 3.70 trans/sec Throughput: 3.70 MB/sec Concurrency: 9.31 Successful transactions: 998 Failed transactions: 2 Longest transaction: 30.09 Shortest transaction: 0.77
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 255.66 secs Data transferred: 1000.00 MB Response time: 2.49 secs Transaction rate: 3.91 trans/sec Throughput: 3.91 MB/sec Concurrency: 9.76 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 10.22 Shortest transaction: 0.72
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 291.95 secs Data transferred: 1000.00 MB Response time: 2.83 secs Transaction rate: 3.43 trans/sec Throughput: 3.43 MB/sec Concurrency: 9.68 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 7.88 Shortest transaction: 0.52
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 1:44 | 25968K |
| LibreSSL 3.2.5 | 1:50 | 23220K |
| LibreSSL devel 3.3.1 | 1:53 | 23796K |
| OpenSSL 1.1.1j,1 | 1:51 | 25668K |
| OpenSSL devel 3.0.0.a13 | 2:03 | 28532K |
| wolfSSL 4.7.0_9 (shared server WOLFSSL_CTX) | 1:59 | 18808K |
| wolfSSL 4.7.0_9 (separate server WOLFSSL_CTX) | 2:06 | 16724K |
| mbedtls 2.16.10_1 | 3:45 | 15356K |
For this test I've enabled siege's keep-alive mode.
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 233.56 secs Data transferred: 1000.00 MB Response time: 2.27 secs Transaction rate: 4.28 trans/sec Throughput: 4.28 MB/sec Concurrency: 9.73 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 6.11 Shortest transaction: 0.23
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 220.82 secs Data transferred: 1000.00 MB Response time: 2.13 secs Transaction rate: 4.53 trans/sec Throughput: 4.53 MB/sec Concurrency: 9.65 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 6.58 Shortest transaction: 0.18
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 208.70 secs Data transferred: 1000.00 MB Response time: 2.02 secs Transaction rate: 4.79 trans/sec Throughput: 4.79 MB/sec Concurrency: 9.66 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 6.71 Shortest transaction: 0.26
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 212.75 secs Data transferred: 1000.00 MB Response time: 2.07 secs Transaction rate: 4.70 trans/sec Throughput: 4.70 MB/sec Concurrency: 9.72 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 7.44 Shortest transaction: 0.23
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 220.55 secs Data transferred: 1000.00 MB Response time: 2.14 secs Transaction rate: 4.53 trans/sec Throughput: 4.53 MB/sec Concurrency: 9.70 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 6.91 Shortest transaction: 0.34
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 213.53 secs Data transferred: 1000.00 MB Response time: 2.07 secs Transaction rate: 4.68 trans/sec Throughput: 4.68 MB/sec Concurrency: 9.69 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 6.02 Shortest transaction: 0.32
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 264.38 secs Data transferred: 1000.00 MB Response time: 2.59 secs Transaction rate: 3.78 trans/sec Throughput: 3.78 MB/sec Concurrency: 9.80 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 7.64 Shortest transaction: 0.37
For this test Privoxy is not overwriting wolfSSL's default receive callback.
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 319.03 secs Data transferred: 1000.00 MB Response time: 3.14 secs Transaction rate: 3.13 trans/sec Throughput: 3.13 MB/sec Concurrency: 9.83 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 12.37 Shortest transaction: 0.36
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --keep-alive --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.0.7 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 265.41 secs Data transferred: 1000.00 MB Response time: 2.56 secs Transaction rate: 3.77 trans/sec Throughput: 3.77 MB/sec Concurrency: 9.66 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 9.44 Shortest transaction: 0.95
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.0.2t | 1:18 | 20108K |
| LibreSSL 3.2.5 | 1:18 | 20104K |
| LibreSSL devel 3.3.1 | 1:19 | 20432K |
| OpenSSL 1.1.1j,1 | 1:24 | 20464K |
| OpenSSL devel 3.0.0.a13 | 1:20 | 22856K |
| wolfSSL 4.7.0_9 (shared server WOLFSSL_CTX) | 2:38 | 23860K |
| wolfSSL 4.7.0_9 (separate server WOLFSSL_CTX) | 1:44 | 14328K |
| wolfSSL 4.7.0_9 (separate server WOLFSSL_CTX, default callback) | 1:39 | 13144K |
| mbedtls 2.16.10_1 | 1:52 | 14204K |
The operating system on the host and in the virtual machines has been changed to ElectroBSD 12.3-STABLE 5c6e955abbf5.
The Privoxy ports have been updated and are close to the 3.0.33 release.
The webserver that serves https://www.electrobsd.org/ has been updated as well and now supports TLS 1.3.
This test is fetching 1 MB of random data with a concurrency level of 10 and 100 reps.
Keep in mind that the network connections of the server hosting www.electrobsd.org and the system hosting the VMs are fairly busy so Privoxy spends a fair amount of time waiting for data to arrive.
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.1.1 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 204.61 secs Data transferred: 1000.00 MB Response time: 1.99 secs Transaction rate: 4.89 trans/sec Throughput: 4.89 MB/sec Concurrency: 9.74 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 7.52 Shortest transaction: 0.52
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.1.1 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 204.48 secs Data transferred: 1000.00 MB Response time: 1.99 secs Transaction rate: 4.89 trans/sec Throughput: 4.89 MB/sec Concurrency: 9.75 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 7.91 Shortest transaction: 0.31
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.1.1 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 259.81 secs Data transferred: 1000.00 MB Response time: 2.54 secs Transaction rate: 3.85 trans/sec Throughput: 3.85 MB/sec Concurrency: 9.79 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 9.87 Shortest transaction: 0.62
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.1.1 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 222.54 secs Data transferred: 1000.00 MB Response time: 2.18 secs Transaction rate: 4.49 trans/sec Throughput: 4.49 MB/sec Concurrency: 9.81 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 7.68 Shortest transaction: 0.35
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.1.1 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 311.33 secs Data transferred: 1000.00 MB Response time: 3.05 secs Transaction rate: 3.21 trans/sec Throughput: 3.21 MB/sec Concurrency: 9.79 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 8.90 Shortest transaction: 0.38
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.1.1 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 179.65 secs Data transferred: 1000.00 MB Response time: 1.75 secs Transaction rate: 5.57 trans/sec Throughput: 5.57 MB/sec Concurrency: 9.72 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 6.98 Shortest transaction: 0.43
[fk@benchmark-vm ~]$ siege --concurrent=10 --reps=100 --benchmark "https://www.electrobsd.org/test/random-data-1M.raw" ** SIEGE 4.1.1 ** Preparing 10 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 273.14 secs Data transferred: 1000.00 MB Response time: 2.57 secs Transaction rate: 3.66 trans/sec Throughput: 3.66 MB/sec Concurrency: 9.42 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 16.87 Shortest transaction: 1.12
| TLS library | Client side TLS version and cipher suite | Server side TLS version and cipher suite |
|---|---|---|
| OpenSSL 1.1.1m | TLSv1.3 (TLS_AES_256_GCM_SHA384) | TLSv1.3 (TLS_AES_256_GCM_SHA384) |
| LibreSSL 3.3.5 | TLSv1.3 (AEAD-AES256-GCM-SHA384) | TLSv1.3 (AEAD-AES256-GCM-SHA384) |
| LibreSSL devel 3.4.0 | TLSv1.3 (AEAD-AES256-GCM-SHA384) | TLSv1.3 (AEAD-AES256-GCM-SHA384) |
| OpenSSL 1.1.1l | TLSv1.3 (TLS_AES_256_GCM_SHA384) | TLSv1.3 (TLS_AES_256_GCM_SHA384) |
| OpenSSL devel 3.0.1 | TLSv1.3 (TLS_AES_256_GCM_SHA384) | TLSv1.3 (TLS_AES_256_GCM_SHA384) |
| wolfSSL 5.0.0 | TLSv1.3 (TLS_AES_256_GCM_SHA384) | TLSv1.3 (TLS13-AES256-GCM-SHA384) |
| mbedtls 2.16.11 | TLSv1.2 (TLS-ECDHE-RSA-WITH-CHACHA20-POLY1305-SHA256) | TLSv1.2 (TLS-ECDHE-ECDSA-WITH-AES-128-GCM-SHA256) |
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.1.1m | 1:49 | 18M |
| LibreSSL 3.3.5 | 1:42 | 17M |
| LibreSSL devel 3.4.0 | 1:42 | 17M |
| OpenSSL 1.1.1l | 1:48 | 19M |
| OpenSSL devel 3.0.1 | 4:02 | 18M |
| wolfSSL 5.0.0 | 1:50 | 13M |
| mbedtls 2.16.11 | 3:17 | 9508K |
There's a new year and thus a new quarterly branch (2022Q1) so some of the TLS backends have been updated.
The Privoxy code has been updated as well and now contains an experimental patch that allows to use elliptic-curve-cryptography instead of RSA when generating keys and Privoxy is compiled against OpenSSL 3.0.X. Doing this has been reported to improve performance on macOS in some situations.
Privoxy has been configured as described above to force key regeneration for each request. Unless noted otherwise, Privoxy is generating 2048 bit RSA keys.
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/' ** SIEGE 4.1.1 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 219.29 secs Data transferred: 20.70 MB Response time: 0.22 secs Transaction rate: 4.56 trans/sec Throughput: 0.09 MB/sec Concurrency: 1.00 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 1.98 Shortest transaction: 0.03
The libressl and libressl-devel ports are both at version 3.4.2 so the test with libressl-devel is skipped.
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/' ** SIEGE 4.1.1 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 259.16 secs Data transferred: 20.70 MB Response time: 0.26 secs Transaction rate: 3.86 trans/sec Throughput: 0.08 MB/sec Concurrency: 1.00 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 1.65 Shortest transaction: 0.09
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/' ** SIEGE 4.1.1 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 213.58 secs Data transferred: 20.70 MB Response time: 0.21 secs Transaction rate: 4.68 trans/sec Throughput: 0.10 MB/sec Concurrency: 1.00 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 1.16 Shortest transaction: 0.03
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/' ** SIEGE 4.1.1 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 698.57 secs Data transferred: 20.70 MB Response time: 0.70 secs Transaction rate: 1.43 trans/sec Throughput: 0.03 MB/sec Concurrency: 1.00 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 3.04 Shortest transaction: 0.19
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/' ** SIEGE 4.1.1 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 38.38 secs Data transferred: 20.70 MB Response time: 0.04 secs Transaction rate: 26.06 trans/sec Throughput: 0.54 MB/sec Concurrency: 0.99 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 0.23 Shortest transaction: 0.03
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/' ** SIEGE 4.1.1 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 158.52 secs Data transferred: 20.70 MB Response time: 0.16 secs Transaction rate: 6.31 trans/sec Throughput: 0.13 MB/sec Concurrency: 1.00 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 0.58 Shortest transaction: 0.04
[fk@benchmark-vm ~]$ siege --concurrent=1 --reps=1000 --benchmark 'https://p.p/user-manual/' ** SIEGE 4.1.1 ** Preparing 1 concurrent users for battle. The server is now under siege... Transactions: 1000 hits Availability: 100.00 % Elapsed time: 342.58 secs Data transferred: 20.70 MB Response time: 0.34 secs Transaction rate: 2.92 trans/sec Throughput: 0.06 MB/sec Concurrency: 1.00 Successful transactions: 1000 Failed transactions: 0 Longest transaction: 2.83 Shortest transaction: 0.08
| TLS library | CPU time | Memory size |
|---|---|---|
| OpenSSL 1.1.1m | 2:48 | 10M |
| LibreSSL 3.4.2 | 2:42 | 8596K |
| OpenSSL 1.1.1m_1,1 | 2:44 | 10M |
| OpenSSL devel 3.0.1 (RSA) | 9:00 | 11M |
| OpenSSL devel 3.0.1 (EC) | 0:26 | 11M |
| wolfSSL 5.1.0 | 2:01 | 11M |
| mbedtls 2.16.12 | 4:16 | 6672K |