HTTPX

4 December 20256 min read

httpx is a fast and multi-purpose HTTP toolkit built to support running multiple probes using a public library. Probes are specific tests or checks to gather information about web servers, URLs, or other HTTP elements. Httpx is designed to maintain result reliability with an increased number of threads.

Usage

Usage: httpx [options]

Options

Input

Option	Description
`-l`, `-list`	input file containing list of hosts to process
`-rr`, `-request`	file containing raw request
`-u`, `-target`	input target host(s) to probe

Probes

Option	Description
`-sc`, `-status-code`	display response status-code
`-cl`, `-content-length`	display response content-length
`-ct`, `-content-type`	display response content-type
`-location`	display response redirect location
`-favicon`	display mmh3 hash for '/favicon.ico' file
`-hash` string	display response body hash (supported: md5, mmh3, simhash, sha1, sha256, sha512)
`-jarm`	display jarm fingerprint hash
`-rt`, `-response-time`	display response time
`-lc`, `-line-count`	display response body line count
`-wc`, `-word-count`	display response body word count
`-title`	display page title
`-bp`, `-body-preview`	display first N characters of response body (default 100)
`-server`, `-web-server`	display server name
`-td`, `-tech-detect`	display technology in use based on wappalyzer dataset
`-method`	display http request method
`-websocket`	display server using websocket
`-ip`	display host ip
`-cname`	display host cname
`-extract-fqdn`, `-efqdn`	get domain and subdomains from response body and header in jsonl/csv output
`-asn`	display host asn information
`-cdn`	display cdn/waf in use (default true)
`-probe`	display probe status

Headless

Option	Description
`-ss`, `-screenshot`	enable saving screenshot of the page using headless browser
`-system-chrome`	enable using local installed chrome for screenshot
`-ho`, `-headless-options`	start headless chrome with additional options
`-esb`, `-exclude-screenshot-bytes`	enable excluding screenshot bytes from json output
`-ehb`, `-exclude-headless-body`	enable excluding headless header from json output
`-no-screenshot-full-page`	disable saving full page screenshot
`-st`, `-screenshot-timeout`	set timeout for screenshot in seconds (default 10s)
`-sid`, `-screenshot-idle`	set idle time before taking screenshot in seconds (default 1s)

Matchers

Option	Description
`-mc`, `-match-code`	match response with specified status code (-mc 200,302)
`-ml`, `-match-length`	match response with specified content length (-ml 100,102)
`-mlc`, `-match-line-count`	match response body with specified line count (-mlc 423,532)
`-mwc`, `-match-word-count`	match response body with specified word count (-mwc 43,55)
`-mfc`, `-match-favicon`	match response with specified favicon hash (-mfc 1494302000)
`-ms`, `-match-string`	match response with specified string (-ms admin)
`-mr`, `-match-regex`	match response with specified regex (-mr admin)
`-mcdn`, `-match-cdn`	match host with specified cdn provider (cloudfront, fastly, gcore, gocache, google)
`-mrt`, `-match-response-time`	match response with specified response time in seconds (-mrt '< 1')
`-mdc`, `-match-condition`	match response with dsl expression condition

Extractor

Option	Description
`-er`, `-extract-regex`	display response content with matched regex
`-ep`, `-extract-preset`	display response content matched by a pre-defined regex (mail, url, ipv4)

Filters

Option	Description
`-fc`, `-filter-code`	filter response with specified status code (-fc 403,401)
`-fep`, `-filter-error-page`	filter response with ML based error page detection
`-fd`, `-filter-duplicates`	filter out near-duplicate responses (only first response is retained)
`-fl`, `-filter-length`	filter response with specified content length (-fl 23,33)
`-flc`, `-filter-line-count`	filter response body with specified line count (-flc 423,532)
`-fwc`, `-filter-word-count`	filter response body with specified word count (-fwc 423,532)
`-ffc`, `-filter-favicon`	filter response with specified favicon hash (-ffc 1494302000)
`-fs`, `-filter-string`	filter response with specified string (-fs admin)
`-fe`, `-filter-regex`	filter response with specified regex (-fe admin)
`-fcdn`, `-filter-cdn`	filter host with specified cdn provider (cloudfront, fastly, gcore, gocache, google)
`-frt`, `-filter-response-time`	filter response with specified response time in seconds (-frt '> 1')
`-fdc`, `-filter-condition`	filter response with dsl expression condition
`-strip`	strips all tags in response. supported formats: html,xml (default html)

Rate-Limit

Option	Description
`-t`, `-threads`	number of threads to use (default 50)
`-rl`, `-rate-limit`	maximum requests to send per second (default 150)
`-rlm`, `-rate-limit-minute`	maximum number of requests to send per minute

Misc

Option	Description
`-pa`, `-probe-all-ips`	probe all the IPs associated with same host
`-p`, `-ports`	ports to probe (nmap syntax: eg http:1,2-10,11,https:80)
`-path`	path or list of paths to probe (comma-separated, file)
`-tls-probe`	send http probes on the extracted TLS domains (dns_name)
`-csp-probe`	send http probes on the extracted CSP domains
`-tls-grab`	perform TLS(SSL) data grabbing
`-pipeline`	probe and display server supporting HTTP1.1 pipeline
`-http2`	probe and display server supporting HTTP2
`-vhost`	probe and display server supporting VHOST
`-ldv`, `-list-dsl-variables`	list json output field keys name that support dsl matcher/filter

Update

Option	Description
`-up`, `-update`	update httpx to latest version
`-duc`, `-disable-update-check`	disable automatic httpx update check

Output

Option	Description
`-o`, `-output`	file to write output results
`-oa`, `-output-all`	filename to write output results in all formats
`-sr`, `-store-response`	store http response to output directory
`-srd`, `-store-response-dir`	store http response to custom directory
`-ob`, `-omit-body`	omit response body in output
`-csv`	store output in csv format
`-csvo`, `-csv-output-encoding`	define output encoding
`-j`, `-json`	store output in JSONL(ines) format
`-irh`, `-include-response-header`	include http response (headers) in JSON output (-json only)
`-irr`, `-include-response`	include http request/response (headers + body) in JSON output (-json only)
`-irrb`, `-include-response-base64`	include base64 encoded http request/response in JSON output (-json only)
`-include-chain`	include redirect http chain in JSON output (-json only)
`-store-chain`	include http redirect chain in responses (-sr only)
`-svrc`, `-store-vision-recon-cluster`	include visual recon clusters (-ss and -sr only)
`-pr`, `-protocol`	protocol to use (unknown, http11)
`-fepp`, `-filter-error-page-path`	path to store filtered error pages (default "filtered_error_page.json")

Configurations

Option	Description
`-config`	path to the httpx configuration file (default $HOME/.config/httpx/config.yaml)
`-r`, `-resolvers`	list of custom resolver (file or comma separated)
`-allow`	allowed list of IP/CIDR's to process (file or comma separated)
`-deny`	denied list of IP/CIDR's to process (file or comma separated)
`-sni`, `-sni-name`	custom TLS SNI name
`-random-agent`	enable Random User-Agent to use (default true)
`-H`, `-header`	custom http headers to send with request
`-http-proxy`, `-proxy`	proxy (http/socks) to use (eg )
`-unsafe`	send raw requests skipping golang normalization
`-resume`	resume scan using resume.cfg
`-fr`, `-follow-redirects`	follow http redirects
`-maxr`, `-max-redirects`	max number of redirects to follow per host (default 10)
`-fhr`, `-follow-host-redirects`	follow redirects on the same host
`-rhsts`, `-respect-hsts`	respect HSTS response headers for redirect requests
`-vhost-input`	get a list of vhosts as input
`-x`	request methods to probe, use 'all' to probe all HTTP methods
`-body`	post body to include in http request
`-s`, `-stream`	stream mode - start elaborating input targets without sorting
`-sd`, `-skip-dedupe`	disable dedupe input items (only used with stream mode)
`-ldp`, `-leave-default-ports`	leave default http/https ports in host header (eg. http://host:80 - https://host:443)
`-ztls`	use ztls library with autofallback to standard one for tls13
`-no-decode`	avoid decoding body
`-tlsi`, `-tls-impersonate`	enable experimental client hello (ja3) tls randomization
`-no-stdin`	Disable Stdin processing
`-hae`, `-http-api-endpoint`	experimental http api endpoint

Debug

Option	Description
`-health-check`, `-hc`	run diagnostic check up
`-debug`	display request/response content in cli
`-debug-req`	display request content in cli
`-debug-resp`	display response content in cli
`-version`	display httpx version
`-stats`	display scan statistic
`-profile-mem`	optional httpx memory profile dump file
`-silent`	silent mode
`-v`, `-verbose`	verbose mode
`-si`, `-stats-interval`	number of seconds to wait between showing statistics update (default: 5)
`-nc`, `-no-color`	disable colors in cli output
`-tr`, `-trace`	trace

Optimizations

Option	Description
`-nf`, `-no-fallback`	display both probed protocol (HTTPS and HTTP)
`-nfs`, `-no-fallback-scheme`	probe with protocol scheme specified in input
`-maxhr`, `-max-host-error`	max error count per host before skipping remaining path/s (default 30)
`-e`, `-exclude`	exclude host matching specified filter ('cdn', 'private-ips', cidr, ip, regex)
`-retries`	number of retries
`-timeout`	timeout in seconds (default 10)
`-delay`	duration between each http request (eg: 200ms, 1s) (default -1ns)
`-rsts`, `-response-size-to-save`	max response size to save in bytes (default 2147483647)
`-rstr`, `-response-size-to-read`	max response size to read in bytes (default 2147483647)

Cloud

Option	Description
`-auth`	configure projectdiscovery cloud (pdcp) api key (default true)
`-ac`, `-auth-config`	configure projectdiscovery cloud (pdcp) api key credential file
`-pd`, `-dashboard`	upload / view output in projectdiscovery cloud (pdcp) UI dashboard
`-tid`, `-team-id`	upload asset results to given team id (optional)
`-aid`, `-asset-id`	upload new assets to existing asset id (optional)
`-aname`, `-asset-name`	assets group name to set (optional)
`-pdu`, `-dashboard-upload`	upload httpx output file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard