[dvd-discuss] Re: Searchable list archives.

[Wendy Seltzer <wendy(at)seltzer.com>]

At 10:14 PM 04/27/2002 -0400, Scott A Crosby wrote:
>One of the things that sorta surprised me was that the openlaw DVD list
>wasn't coming up on any of my searches of 'Vivian's Vow'. I would have
>expected google to have indexed the archives by now and searches for
>'Sara Glover' or 'Vivian's Vow' to be returning our messages.
>
>But its not.
>
>I checked out  http://cyber.law.harvard.edu/robots.txt

...

>And I think that this policy should be changed. Google-searchable list
>archives are a valuabale resource for ourselves, for others, and can serve
>as an advertisement of our cause.
>
>So.. Whats the verdict about changing robots.txt to allow search engines
>(especially google) to index the list archives?

I'm happy to change the robots.txt -- the restrictions were more relevant 
to other lists archived there, from which people thought it strange to see 
list messages appearing out of context in searches.   You're right though, 
there's plenty of discussion on dvd-discuss that might be of general 
interest. (if perhaps not to Vivian...)

Any objections?

--Wendy


--
Wendy Seltzer -- wendy@seltzer.com
Fellow, Berkman Center for Internet & Society at Harvard Law School
http://cyber.law.harvard.edu/seltzer.html
Chilling Effects: http://www.chillingeffects.org/