[ipac] HIP Use Count statistics-robots.txt-Part 2

[Jonathan Rochkind]

It doesn't bother me if people out there are looking at my public 
server. It's a public server. It only bothers me when it's causing a 
problem of some kind---making it hard to analyze my stats, causing more 
traffic than my server can handle, etc. In which case when the problem 
arises, I try to find ways to deal with it. It can indeed be a challenge 
sometimes.

But, indeed "As long as a web server is reachable from the Internet, as 
long as there is a pathway to it, it is going to get hit by all kinds of 
stuff."  This doesn't alarm me at all! It's the nature of the beast. 
Looking at my logs and seeing that I was getting 'probed' would neither 
surprise nor alarm me. My public servers are meant to be public. My 
internal servers are protected by firewalls.

Jonathan


Vern Mastel wrote:
> At 10:59 AM 3/28/2008 -0400, you wrote:
>   
>> I put a robots.txt on my link resolver server, and immediately saw
>> Google and Yahoo traffic dissappear, no problem.  Haven't thought about
>> it/tried it with HIP yet.  But if you have other experience, then I
>> guess I'm just glad I don't, that would indeed be a pain to deal with!
>> It wouldn't surprise me that there are other less scrupulous robots out
>> there though (including some that are looking for email addresses to
>> spam, security breaches in your web server, etc.).   I haven't yet run
>> into any serious problems with them yet myself, fortunately.
>>
>> Jonathan
>>     
>
> As long as a web server is reachable from the Internet, as long as there is 
> a pathway to it, it is going to get hit by all kinds of stuff.
>
> If it can be found, it will be indexed, attacked, probed and in general, 
> dorked with.
>
> I almost stopped reading my server logs, it was making me paranoid.
>
> Recently I setup a new web server but did not add it to the external DNS so 
> it could not be found by name from the Internet.
>
> However when looking in the log file after a week, I was quite surprised to 
> see that it had been found and probed by several IPs out on the Internet.
>
> Yikes.
>
>
> Vern Mastel
> Technology Coordinator, Bismarck Veterans Memorial Public Library
> Desk Phone 701-355-1499 Cell Phone 701-426-5897
>
>
> _______________________________________________
> ipac mailing list
> ipac at lists.tblc.org
> http://lists.tblc.org/mailman/listinfo/ipac
>
>   

-- 
Jonathan Rochkind
Digital Services Software Engineer
The Sheridan Libraries
Johns Hopkins University
410.516.8886 
rochkind (at) jhu.edu