summaryrefslogtreecommitdiff
path: root/content/fetchers/curl.c
Commit message (Collapse)AuthorAgeFilesLines
* Fetch/curl: expose socket open/close via fetch vtableJohn-Mark Bell2024-03-131-0/+20
| | | | | This allows frontends to customise the behaviour of sockets. The default implementation simply maps to socket(2)/close(2).
* fetchers: drop support for ancient OpenSSLJohn-Mark Bell2024-03-131-25/+3
| | | | | | | | | We now require 1.1.1 or later (and, preferably, 3.x). Also take the opportunity to add support for LibreSSL 3.5.0 or later (we still support ancient versions of this because 3.5.0 is still relatively modern -- give it a few more years and support for old LibreSSL can go, too)
* Revert "fetchers: drop support for ancient OpenSSL"John-Mark Bell2024-02-111-3/+25
| | | | | | | ciworker{8,12} (respectively: FreeBSD, aarch64 Linux) are running obsolete OS versions. Disappointment ensues. This reverts commit 6bb70e88108c904d67e9af7c8e5b273f6cd6854f.
* fetchers: drop support for ancient OpenSSLJohn-Mark Bell2024-02-081-25/+3
| | | | | | | | | We now require 1.1.1 or later (and, preferably, 3.x). Also take the opportunity to add support for LibreSSL 3.5.0 or later (we still support ancient versions of this because 3.5.0 is still relatively modern -- give it a few more years and support for old LibreSSL can go, too)
* fetchers/curl: modernise TLS 1.2 cipher suitesJohn-Mark Bell2024-02-041-8/+25
| | | | | | | | | * Drop support for DHE completely (logjam plus compat woes caused other browsers to do this some time ago) * Minimise CBC-mode suites * Fall back to non-PFS RSA suites if really necessary (we treat this as a protocol downgrade as anything modern should either be using TLS 1.3 or have support for the ECDHE suites)
* Fix use of uninitialised code variable in error reportingVincent Sanders2023-12-131-3/+4
|
* Attempt to fix maybe uninitialised errorVincent Sanders2023-12-131-1/+1
|
* Resolve progress function deprication in curl later than 7.32.0Vincent Sanders2023-12-121-6/+16
|
* Remove use of depricated API after libcurl 7.56Vincent Sanders2023-12-121-122/+401
|
* fetchers/curl: tolerate lack of TLS1.3John-Mark Bell2023-03-051-2/+6
| | | | | | | | If we are building against a modern version of libcurl, but it was built against a version of OpenSSL that does not support TLS1.3, then attempting to configure TLS1.3 ciphersuites will fail with CURLE_NOT_BUILT_IN. Tolerate this scenario by treating such a return code as non-fatal in this case.
* (curl): Add extra logging to fetcher initialisationDaniel Silverstone2022-12-151-4/+8
| | | | Signed-off-by: Daniel Silverstone <daniel.silverstone@codethink.co.uk>
* fetchers/curl: drop non-PFS fallback ciphersuiteJohn-Mark Bell2022-11-051-2/+0
|
* fetchers/curl: disable TLS1.0 and TLS1.1 supportJohn-Mark Bell2022-11-051-10/+6
|
* fetchers/curl: explicitly configure TLS1.3 ciphersuitesJohn-Mark Bell2022-11-051-1/+13
| | | | These currently match the defaults, so no functional change.
* hoist the Referer header generation logic up to the low level cacheVincent Sanders2021-03-251-1/+0
| | | | | | | This removes the need for the fetchers to have any interaction with the Referer header. It has not been completely removed from the fetch interface as fetch.c:fetch_set_cookie() still uses it for unverifiable cookie decision logic. (There is an anchient todo here)
* fetchers/curl: Move report_certs function outside WITH_OPENSSLDaniel Silverstone2020-05-221-21/+22
| | | | Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* curl: Pass cert chain on even if we get no headersDaniel Silverstone2020-05-041-0/+4
| | | | Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* Curl fetcher: Guard against read of uninitialised value.Michael Drake2020-02-241-0/+3
| | | | | | | Fixes Coverity issue: CID 1419830: Memory - illegal accesses (UNINIT) >>> Using uninitialized value "buf[depth]".
* Curl fetcher: Squash coverity warning.Michael Drake2020-02-241-3/+2
| | | | | CID 1419832: Control flow issues (DEADCODE) >>> Execution cannot reach this statement: "goto out;".
* curl: Implement SSL chain cache in cURL fetcherDaniel Silverstone2020-02-231-8/+150
| | | | | | | | | | | Because cURL can do connection caching behind the scenes, we need to have a cache for the SSL certificate chains which we send onward on first header back from cURL. This uses the new hashmap implementation to mean that we cache chains on a hostname:port basis. Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* Keep the complete certificate chain from a fetchVincent Sanders2020-02-231-120/+57
| | | | | | | | | Instead of extracting information from the X509 certificate chain in the fetcher the entire chain is propagated in Distinguished Encoding Rules (DER) format. This allows all the information contained in a certificate chain to be retained which can subsequently be presented to the user
* fetch_curl_report_certs_upstream: Use new SSL_CERT_ERR_CERT_MISSINGDaniel Silverstone2019-12-031-1/+5
| | | | Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* fetch_curl_verify_callback: Do depth update after checkDaniel Silverstone2019-12-031-5/+5
| | | | Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* remove user warnings from hotlist load and curl pollVincent Sanders2019-11-051-2/+2
|
* fetchers/curl: Restrict AUTH to BASICDaniel Silverstone2019-10-211-1/+1
| | | | | | | | | | | | | cURL will prevent channel reuse if NTLM auth is enabled because NTLM authenticates a channel not a request. As such we were unable to reuse curl handles since we handed off connection reuse to curl instead of our own handle cache. This mitigates the effect, though curl authors are looking at fixing it upstream too. Fixes: #2707 Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* Use curl API (versions after 7.56.0) to determine if openssl is in useVincent Sanders2019-10-011-4/+23
|
* Allow the curl fetcher to be built without openssl.Vincent Sanders2019-09-301-30/+61
| | | | | | | | | | The curl fetcher can operate without openssl library being available, additionaly curl itself may be compiled with a different TLS library. In either case this will simply cause the "unknown" error to be reported for all TLS failiures and page information to lack any certificate information.
* SSL Error: Enable OpenSSL hostname verificationDaniel Silverstone2019-08-141-16/+19
| | | | | | | | Since OpenSSL 1.0.2 there has been hostname verification support which cURL doesn't turn on for some reason. Turn it on so that we get better hostname verification handling. Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* add common name ssl certificate errorVincent Sanders2019-08-101-3/+16
| | | | | This adds an ssl faliure code and explanation why curl fetcher does not currently set it.
* Support SSL verification through new about: handlerDaniel Silverstone2019-08-061-0/+43
| | | | | | | | In doing this, also propagate why the certificates were bad so that the page can display a reason. We will need FatMessages for all these. Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* curl.c: Don't use OpenSSL 1.1 only serialNumber fetcherDaniel Silverstone2019-08-051-1/+1
| | | | Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* Migrate SSL certificate storage to the browser windowDaniel Silverstone2019-08-051-119/+132
| | | | | | | | | | * Fetchers now provide the certificates before headers * This is propagated all the way to the browser window * When a query occurs, we retrieve it from there and fire the query with those stored certificates. * The serial number is a bignum, store it as hex. Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* curl: Change where we lodge `inside_curl`Daniel Silverstone2019-08-021-6/+2
| | | | | | | | Because we need to be slightly more careful now, lodge ourselves `inside_curl` whenever we might be processing via the cURL messages or similar. Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* curl: Immediately abort fetchers when we canDaniel Silverstone2019-08-021-17/+24
| | | | Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* content/fetchers/curl.c: Defer fetch start if inside cURLDaniel Silverstone2019-06-051-0/+16
| | | | | | | | In order to cope better with modern cURL which prevents making cURL calls when inside a callback from within cURL, defer fetch start when we are processing in `fetch_curl_data()`. Signed-off-by: Daniel Silverstone <dsilvers@digital-scurf.org>
* make curl fetcher log debug using NSLOGVincent Sanders2019-05-281-16/+26
| | | | | | | stop curl fetcher logging being special case to standard error and use the fetch catagory at DEBUG level instead. The special suppress_curl_debug option is currently still obeyed
* Curl fetcher: Force HTTP1.1 to fix HTTP2 crash.Michael Drake2019-05-061-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On ubuntu 19.4, curl is built with HTTP2 support, and we segfault. ==18174== Invalid read of size 1 ==18174== at 0x4ACCE7D: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0) ==18174== by 0x4B054B1: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0) ==18174== by 0x4AD398A: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0) ==18174== by 0x4AD7A0B: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0) ==18174== by 0x4AE93EE: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0) ==18174== by 0x4AEA8A8: curl_multi_perform (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0) ==18174== by 0x1F2EF7: fetch_curl_poll (curl.c:1209) ==18174== by 0x1EEC5C: fetcher_poll (fetch.c:271) ==18174== by 0x2A1ED4: schedule_run (schedule.c:160) ==18174== by 0x15F941: framebuffer_run (gui.c:596) ==18174== by 0x15F941: main (gui.c:2206) ==18174== Address 0x9de95a8 is 3,224 bytes inside a block of size 6,304 free'd ==18174== at 0x483997B: free (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==18174== by 0x4AD497B: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0) ==18174== by 0x4AE158C: curl_easy_cleanup (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0) ==18174== by 0x1F30DE: fetch_curl_cache_handle (curl.c:761) ==18174== by 0x1F30DE: fetch_curl_stop (curl.c:840) ==18174== by 0x1F30DE: fetch_curl_done (curl.c:1122) ==18174== by 0x1F30DE: fetch_curl_poll (curl.c:1223) ==18174== by 0x1EEC5C: fetcher_poll (fetch.c:271) ==18174== by 0x2A1ED4: schedule_run (schedule.c:160) ==18174== by 0x15F941: framebuffer_run (gui.c:596) ==18174== by 0x15F941: main (gui.c:2206) ==18174== Block was alloc'd at ==18174== at 0x483AB35: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==18174== by 0x4AE165F: curl_easy_duphandle (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0) ==18174== by 0x1F15EB: fetch_curl_get_handle (curl.c:738) ==18174== by 0x1F15EB: fetch_curl_start (curl.c:750) ==18174== by 0x1EEB22: fetch_dispatch_job (fetch.c:156) ==18174== by 0x1EEB22: fetch_choose_and_dispatch (fetch.c:187) ==18174== by 0x1EEB22: fetch_dispatch_jobs (fetch.c:247) ==18174== by 0x1EF1BB: fetch_start (fetch.c:573) ==18174== by 0x26C779: llcache_object_refetch (llcache.c:916) ==18174== by 0x26D5E4: llcache_object_fetch (llcache.c:979) ==18174== by 0x26D5E4: llcache_object_retrieve_from_cache (llcache.c:1767) ==18174== by 0x26D5E4: llcache_object_retrieve (llcache.c:1865) ==18174== by 0x26E42C: llcache_fetch_redirect (llcache.c:2110) ==18174== by 0x26E42C: llcache_fetch_callback (llcache.c:2810) ==18174== by 0x1F1295: fetch_curl_process_headers (curl.c:922) ==18174== by 0x1F13A0: fetch_curl_data (curl.c:1324) ==18174== by 0x4ACD4C3: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0) ==18174== by 0x4AE00DA: ??? (in /usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0)
* Curl fetcher: Jiggle logging levels.Michael Drake2018-08-091-2/+2
|
* Curl fetcher: Move poll logging from INFO to DEEPDEBUG.Michael Drake2018-08-091-2/+2
|
* HTTPS: restrict ciphersuitesJohn-Mark Bell2018-04-221-0/+18
|
* Explicitly re-enable TLS session tickets.John-Mark Bell2018-04-221-0/+4
| | | | Libcurl disables this by default.
* Force SSL session caching on, Ubuntu seems to have it off by defaultDaniel Silverstone2018-04-221-0/+3
|
* Revert "Disable SSL session ID caching."Daniel Silverstone2018-04-221-3/+0
| | | | This reverts commit debe0e345bf453a58bf319a84d5845cce6b38d8e.
* Use coccinelle to change logging macro calls in c filesVincent Sanders2017-09-061-31/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | for F in $(git ls-files '*.c');do spatch --sp-file foo.cocci --in-place ${F};done @@ expression E; @@ -LOG(E); +NSLOG(netsurf, INFO, E); @@ expression E, E1; @@ -LOG(E, E1); +NSLOG(netsurf, INFO, E, E1); @@ expression E, E1, E2; @@ -LOG(E, E1, E2); +NSLOG(netsurf, INFO, E, E1, E2); @@ expression E, E1, E2, E3; @@ -LOG(E, E1, E2, E3); +NSLOG(netsurf, INFO, E, E1, E2, E3); @@ expression E, E1, E2, E3, E4; @@ -LOG(E, E1, E2, E3, E4); +NSLOG(netsurf, INFO, E, E1, E2, E3, E4); @@ expression E, E1, E2, E3, E4, E5; @@ -LOG(E, E1, E2, E3, E4, E5); +NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5); @@ expression E, E1, E2, E3, E4, E5, E6; @@ -LOG(E, E1, E2, E3, E4, E5, E6); +NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5, E6); @@ expression E, E1, E2, E3, E4, E5, E6, E7; @@ -LOG(E, E1, E2, E3, E4, E5, E6, E7); +NSLOG(netsurf, INFO, E, E1, E2, E3, E4, E5, E6, E7);
* Fix OpenSSL X509 cert API detectionVincent Sanders2016-11-211-2/+4
| | | | | LibreSSL defines a OPENSSL_VERSION_NUMBER of 2.1 but only provides the old OpenSSL 1.0.x API.
* fix openSSL 1.1.0 X509 certificate handlingVincent Sanders2016-11-201-9/+23
|
* Refactor the fdset acquisition into the fetchers to stop fetch.c including ↵Daniel Silverstone2016-06-271-0/+18
| | | | curl.h
* Fix handling of certificate chainsVincent Sanders2016-06-271-145/+251
| | | | | | | When processing a x509 certificate chain from openssl it is necessary to allow teh entire chain to be processed rather than halting processing at the first certificate with an error. This allows errors with a certificate authority to be examined.
* move fetch header into public APIVincent Sanders2016-05-301-1/+1
|
* move misc header into public APIVincent Sanders2016-05-301-1/+1
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-04-20 02:45:28 +0000