Robots.txt: Difference between revisions
From DWIKI
m →Links Tag: wikieditor |
Tag: wikieditor |
||
| Line 2: | Line 2: | ||
==bad bots== | ==bad bots== | ||
*[https://www.humansecurity.com/learn/blog/crawlers-list-known-bots-guide/ List of tool/bots you might want to block] | *[https://www.humansecurity.com/learn/blog/crawlers-list-known-bots-guide/ List of tool/bots you might want to block] | ||
*[https://idiallo.com/blog/zipbomb-protection zipbomb protection] | |||
==Documentation== | ==Documentation== | ||
