Letting your site be indexed, but restricting 'reading' of posts to members?

Discussion in 'Forum SEO' started by jilly, May 8, 2004.

  1. jilly

    jilly Fan

    As of right now, the posts on our board are hidden from search engine spiders - I am considering making them visible, so it would be easier for people searching to find the board - however, is there a way to let the board be indexed, but if someone clicks on a link from google to read a post, they have to register before they can read it?
  2. FansofRealityTV

    FansofRealityTV Habitué

    Not that I'm aware of. Spiders are just like guests, and have the same access. You can't give a spider access, and not a guest. And even so, someone searching Google could just hit "View cached version" and see the whole thing, without hitting your site.
  3. rex_b

    rex_b Fanatic


    Well I think you could. There are cloaking techniques that allow you to serve spiders whatever information you want them to see.

    Also you could set the IP of the spiders to be members, so when they come onto your site they are fed member stuff instead of guests.
  4. pablo

    pablo freundlich

    just leave it open access when the spiders are around and then close it when they are off spinning their webs.
  5. Lugh

    Lugh Habitué

    I wonder could you make the 'description' of the spiders--which can be entered in Admin CP--a member?
  6. FansofRealityTV

    FansofRealityTV Habitué

    You can't make a spider a "member", regardless of the IP or description, because you can't pass it a cookie, and you can't make it "log in" the first time.
  7. MaXBoost

    MaXBoost Adherent

    lol, come spider you know you wanna be a member :) :maggie:

  8. Lugh

    Lugh Habitué

    The itsy-bitsy one :D
  9. Tombot

    Tombot Enthusiast

    I think there is a way to do it with mod-rewrite in .htaccess, because I've seen private membership sites do it. Couple of things though. One, like FORT said, if the pages are cached in Google, users can see them that way. Two, I think if the people at Google catch on to what you are doing, they won't like it at all. Their users will be expecting certain content, when all they really get is your no permission page. Just my opinion though, I don't know for a fact that they won't like it.
  10. quentin

    quentin Habitué

    have to agree with tombot, if google visits your pages, they will be cached and so people will see it in the cache. cloaking can indeed lead to being banned if one of your competitor reports you. and finally, as a user, if i search for something in google and arrive to a register page, i will just go back to serps and open the next link...
  11. Jon

    Jon Enthusiast

    But you can make certain pages invisible to guests/spiders like a private forum of some kind, I will pretty much be doing the same thing. There's just some things that will be posted that prying eyes won't need to see unless they choose to register.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.