Maker tech hub — AI · 3D print · Pi · ESP32 · plus the classic tech archive.

Free ESP32 kit · Books · Network Ninja · Archive

Avery J. Parker Practical maker tech — AI tools, 3D printing, Raspberry Pi & ESP32
Classic tech archive. From the original averyjparker.com tech blog — historical context; pair with modern guides where noted. Full archive · Maker projects · Network Ninja

Classic tip · Classic

Wget user agent avoidance

· · Computers , Linux , Linux Software , Linux Tech Support

I use wget a LOT. Many times in working with a website I've got a ssh console opened to the site and if there's a plugin that I'd like to download it doesn't make sense to download and then upload, I…

Written by

Avery J. Parker

IT veteran, maker educator, and author of Network Ninja, 3D Printing Mastery, and AI Workflow Mastery. Business IT: Diversified Tech Solutions.

I use wget a LOT. Many times in working with a website I've got a ssh console opened to the site and if there's a plugin that I'd like to download it doesn't make sense to download and then upload, I want to just download it directly from xyz.com webserver to the website where it will be installed. However, many times people have set up rules to block wget downloads. Yes, I know people use wget to suck down entire websites, it uses up bandwidth, etc. etc. Well. Wget is capable of sending a different user-agent.


First off, the way you are tipped off that wget is disliked is if you've just visited the site and then try wget and get a message like this...

"HTTP request sent, awaiting response... 403 Forbidden"

essentially, if you add --user-agent=MSIE it might bypass user-agent filtering.

Related