Wget
From DWIKI
HOWTO
Mirror entire site
wget --mirror --convert-links --adjust-extension --page-requisites --no-parent https://site-to-download.com
FAQ
Will not follow any links on this page
Try -e robots=off
wget --mirror --convert-links --adjust-extension --page-requisites --no-parent https://site-to-download.com
Try -e robots=off