HTTPX

httpx is a fast and multi-purpose HTTP toolkit built to support running multiple probes using a public library. Probes are specific tests or checks to gather information about web servers, URLs, or other HTTP elements. Httpx is designed to maintain result reliability with an increased number of threads.

Usage

Usage: httpx [options]

Options

Input

Option	Description
-l, -list	input file containing list of hosts to process
-rr, -request	file containing raw request
-u, -target	input target host(s) to probe

Probes

Option	Description
-sc, -status-code	display response status-code
-cl, -content-length	display response content-length
-ct, -content-type	display response content-type
-location	display response redirect location
-favicon	display mmh3 hash for ‘/favicon.ico’ file
-hash string	display response body hash (supported: md5, mmh3, simhash, sha1, sha256, sha512)
-jarm	display jarm fingerprint hash
-rt, -response-time	display response time
-lc, -line-count	display response body line count
-wc, -word-count	display response body word count
-title	display page title
-bp, -body-preview	display first N characters of response body (default 100)
-server, -web-server	display server name
-td, -tech-detect	display technology in use based on wappalyzer dataset
-method	display http request method
-websocket	display server using websocket
-ip	display host ip
-cname	display host cname
-extract-fqdn, -efqdn	get domain and subdomains from response body and header in jsonl/csv output
-asn	display host asn information
-cdn	display cdn/waf in use (default true)
-probe	display probe status

Headless

Option	Description
-ss, -screenshot	enable saving screenshot of the page using headless browser
-system-chrome	enable using local installed chrome for screenshot
-ho, -headless-options	start headless chrome with additional options
-esb, -exclude-screenshot-bytes	enable excluding screenshot bytes from json output
-ehb, -exclude-headless-body	enable excluding headless header from json output
-no-screenshot-full-page	disable saving full page screenshot
-st, -screenshot-timeout	set timeout for screenshot in seconds (default 10s)
-sid, -screenshot-idle	set idle time before taking screenshot in seconds (default 1s)

Matchers

Option	Description
-mc, -match-code	match response with specified status code (-mc 200,302)
-ml, -match-length	match response with specified content length (-ml 100,102)
-mlc, -match-line-count	match response body with specified line count (-mlc 423,532)
-mwc, -match-word-count	match response body with specified word count (-mwc 43,55)
-mfc, -match-favicon	match response with specified favicon hash (-mfc 1494302000)
-ms, -match-string	match response with specified string (-ms admin)
-mr, -match-regex	match response with specified regex (-mr admin)
-mcdn, -match-cdn	match host with specified cdn provider (cloudfront, fastly, gcore, gocache, google)
-mrt, -match-response-time	match response with specified response time in seconds (-mrt ’< 1’)
-mdc, -match-condition	match response with dsl expression condition

Extractor

Option	Description
-er, -extract-regex	display response content with matched regex
-ep, -extract-preset	display response content matched by a pre-defined regex (mail, url, ipv4)

Filters

Option	Description
-fc, -filter-code	filter response with specified status code (-fc 403,401)
-fep, -filter-error-page	filter response with ML based error page detection
-fd, -filter-duplicates	filter out near-duplicate responses (only first response is retained)
-fl, -filter-length	filter response with specified content length (-fl 23,33)
-flc, -filter-line-count	filter response body with specified line count (-flc 423,532)
-fwc, -filter-word-count	filter response body with specified word count (-fwc 423,532)
-ffc, -filter-favicon	filter response with specified favicon hash (-ffc 1494302000)
-fs, -filter-string	filter response with specified string (-fs admin)
-fe, -filter-regex	filter response with specified regex (-fe admin)
-fcdn, -filter-cdn	filter host with specified cdn provider (cloudfront, fastly, gcore, gocache, google)
-frt, -filter-response-time	filter response with specified response time in seconds (-frt ’> 1’)
-fdc, -filter-condition	filter response with dsl expression condition
-strip	strips all tags in response. supported formats: html,xml (default html)

Rate-Limit

Option	Description
-t, -threads	number of threads to use (default 50)
-rl, -rate-limit	maximum requests to send per second (default 150)
-rlm, -rate-limit-minute	maximum number of requests to send per minute

Misc

Option	Description
-pa, -probe-all-ips	probe all the IPs associated with same host
-p, -ports	ports to probe (nmap syntax: eg http:1,2-10,11,https:80)
-path	path or list of paths to probe (comma-separated, file)
-tls-probe	send http probes on the extracted TLS domains (dns_name)
-csp-probe	send http probes on the extracted CSP domains
-tls-grab	perform TLS(SSL) data grabbing
-pipeline	probe and display server supporting HTTP1.1 pipeline
-http2	probe and display server supporting HTTP2
-vhost	probe and display server supporting VHOST
-ldv, -list-dsl-variables	list json output field keys name that support dsl matcher/filter

Update

Option	Description
-up, -update	update httpx to latest version
-duc, -disable-update-check	disable automatic httpx update check

Output

Option	Description
-o, -output	file to write output results
-oa, -output-all	filename to write output results in all formats
-sr, -store-response	store http response to output directory
-srd, -store-response-dir	store http response to custom directory
-ob, -omit-body	omit response body in output
-csv	store output in csv format
-csvo, -csv-output-encoding	define output encoding
-j, -json	store output in JSONL(ines) format
-irh, -include-response-header	include http response (headers) in JSON output (-json only)
-irr, -include-response	include http request/response (headers + body) in JSON output (-json only)
-irrb, -include-response-base64	include base64 encoded http request/response in JSON output (-json only)
-include-chain	include redirect http chain in JSON output (-json only)
-store-chain	include http redirect chain in responses (-sr only)
-svrc, -store-vision-recon-cluster	include visual recon clusters (-ss and -sr only)
-pr, -protocol	protocol to use (unknown, http11)
-fepp, -filter-error-page-path	path to store filtered error pages (default “filtered_error_page.json”)

Configurations

Option	Description
-config	path to the httpx configuration file (default $HOME/.config/httpx/config.yaml)
-r, -resolvers	list of custom resolver (file or comma separated)
-allow	allowed list of IP/CIDR’s to process (file or comma separated)
-deny	denied list of IP/CIDR’s to process (file or comma separated)
-sni, -sni-name	custom TLS SNI name
-random-agent	enable Random User-Agent to use (default true)
-H, -header	custom http headers to send with request
-http-proxy, -proxy	proxy (http/socks) to use (eg http://127.0.0.1:8080)
-unsafe	send raw requests skipping golang normalization
-resume	resume scan using resume.cfg
-fr, -follow-redirects	follow http redirects
-maxr, -max-redirects	max number of redirects to follow per host (default 10)
-fhr, -follow-host-redirects	follow redirects on the same host
-rhsts, -respect-hsts	respect HSTS response headers for redirect requests
-vhost-input	get a list of vhosts as input
-x	request methods to probe, use ‘all’ to probe all HTTP methods
-body	post body to include in http request
-s, -stream	stream mode - start elaborating input targets without sorting
-sd, -skip-dedupe	disable dedupe input items (only used with stream mode)
-ldp, -leave-default-ports	leave default http/https ports in host header (eg. http://host:80 - https://host:443)
-ztls	use ztls library with autofallback to standard one for tls13
-no-decode	avoid decoding body
-tlsi, -tls-impersonate	enable experimental client hello (ja3) tls randomization
-no-stdin	Disable Stdin processing
-hae, -http-api-endpoint	experimental http api endpoint

Debug

Option	Description
-health-check, -hc	run diagnostic check up
-debug	display request/response content in cli
-debug-req	display request content in cli
-debug-resp	display response content in cli
-version	display httpx version
-stats	display scan statistic
-profile-mem	optional httpx memory profile dump file
-silent	silent mode
-v, -verbose	verbose mode
-si, -stats-interval	number of seconds to wait between showing statistics update (default: 5)
-nc, -no-color	disable colors in cli output
-tr, -trace	trace

Optimizations

Option	Description
-nf, -no-fallback	display both probed protocol (HTTPS and HTTP)
-nfs, -no-fallback-scheme	probe with protocol scheme specified in input
-maxhr, -max-host-error	max error count per host before skipping remaining path/s (default 30)
-e, -exclude	exclude host matching specified filter (‘cdn’, ‘private-ips’, cidr, ip, regex)
-retries	number of retries
-timeout	timeout in seconds (default 10)
-delay	duration between each http request (eg: 200ms, 1s) (default -1ns)
-rsts, -response-size-to-save	max response size to save in bytes (default 2147483647)
-rstr, -response-size-to-read	max response size to read in bytes (default 2147483647)

Cloud

Option	Description
-auth	configure projectdiscovery cloud (pdcp) api key (default true)
-ac, -auth-config	configure projectdiscovery cloud (pdcp) api key credential file
-pd, -dashboard	upload / view output in projectdiscovery cloud (pdcp) UI dashboard
-tid, -team-id	upload asset results to given team id (optional)
-aid, -asset-id	upload new assets to existing asset id (optional)
-aname, -asset-name	assets group name to set (optional)
-pdu, -dashboard-upload	upload httpx output file (jsonl) in projectdiscovery cloud (pdcp) UI dashboard