Wget
From DWIKI
HOWTO
Mirror entire site
wget --mirror --convert-links --adjust-extension --page-requisites --no-parent https://site-to-download.com
FAQ
Will not follow any links on this page
Try -e robots=off
Output to stdout
wget -q -O - http://example.com
