User Tools

Site Tools


linux:wget

Linux » wget

Mirror com temporização entre requests, useragent forging e desrespeito por robots.txt

   wget -e robots=off --wait 5 -m -U Googlebot http://foo.com
linux/wget.txt · Last modified: 2010/12/03 19:22 by ambs