Home › curl user agent switch

curl user agent switch 

voodoo
Joined:
04/07/2009
Posts:
163

July 21, 2010 16:10:47    Last update: July 21, 2010 16:10:47
Google news RSS does not like curl: 
C:\> curl --dump-header - -o NUL http://news.google.com/?output=rss
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0HTTP/1.1 403 Forbidden
Content-Type: text/html; charset=UTF-8
Set-Cookie: PREF=ID=d25a6334e29f8a90:TM=1279728231:LM=1279728231:S=cXfnyhgZ8VIZuq13; expires=Fri, 20-Jul-2012 16:03:51 GMT; path=/; domain=.google.com
X-Content-Type-Options: nosniff
Date: Wed, 21 Jul 2010 16:03:51 GMT
Server: NFE/1.0
Content-Length: 1383
X-XSS-Protection: 1; mode=block

100  1383  100  1383    0     0   8135      0 --:--:-- --:--:-- --:--:-- 23440


Switch the agent to HotJava with the -A option: 
C:\> curl -A "HotJava/1.1.2 FCS" --dump-header - -o google-news.xmn http://news.google.com/?output=rss
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0HTTP/1.1 200 OK
Content-Type: application/xml; charset=UTF-8
Date: Wed, 21 Jul 2010 16:08:06 GMT
Expires: Wed, 21 Jul 2010 16:08:06 GMT
Cache-Control: private, max-age=0
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Set-Cookie: PREF=ID=9b9d5007a1eb0aae:TM=1279728486:LM=1279728486:S=hQ6GSeHfdGg8n
fjr; expires=Fri, 20-Jul-2012 16:08:06 GMT; path=/; domain=.google.com
Content-Length: 50216

100 50216  100 50216    0     0   120k      0 --:--:-- --:--:-- --:--:--  155k
Share |
| Comment  | Tags
 

Related Notes