Active TopicsActive Topics  Display List of Forum MembersMemberlist  HelpHelp   RegisterRegister  LoginLogin
ISAPI_Rewrite 3.0 support forum
 Helicon Tech : ISAPI_Rewrite 3.0 support forum
Subject Topic: Duplicate content with Googgle Post ReplyPost New Topic
Author
Message << Prev Topic | Next Topic >>
mamin123
Newbie
Newbie


Joined: 27 August 2008
Online Status: Offline
Posts: 16
Posted: 22 September 2008 at 10:10pm | IP Logged Quote mamin123

Lately i have implemented entire site with URL rewriting using ISAPI Rewrite and during this process by mistake google has create over 1000 pages with duplicate Title tag and description tag

e.g

/carrom-large-nokhockey-set-p-ccm20.htm

/carrom-large-nokhockeyset-p-ccm20.htm

contains the same title and description tag. This happened due to some changes I made in generating urls as you can see above. Now I am not sure how do I remove one of the page for such 1000s diff. urls. Both URL will work fine since it has required parameter with my rules

Is there any way to do 301 redirect for 1000s of page where all pages are different ? just to do clean up ?

 

I am hoping someone gives me some idea using ISAPI rewrite too which allows me to do this. What if I can generate one .txt file with from and to url for 301 redirect and use ISAPI rewrite rule somehow to do this..please help me with required rule

Thanks

 



Edited by mamin123 - 22 September 2008 at 10:11pm
Back to Top View mamin123's Profile Search for other posts by mamin123
 
Anton
Moderator Group
Moderator Group


Joined: 30 January 2007
Location: Ukraine
Online Status: Offline
Posts: 4701
Posted: 23 September 2008 at 3:29am | IP Logged Quote Anton

The best way is to use map file containing from and to urls.
e.g.:
mapfile.txt
carrom-large-nokhockey-set-p-ccm20.htm carrom-large-nokhockeyset-p-ccm20.htm
...

Config:
RewriteEngine on
RewriteBase /
RewriteMap mapfile txt:mapfile.txt
RewriteRule ^(.+)$ /${mapfile:$1} [R=301,L]


__________________
Regards,
Anton
Back to Top View Anton's Profile Search for other posts by Anton
 
mamin123
Newbie
Newbie


Joined: 27 August 2008
Online Status: Offline
Posts: 16
Posted: 23 September 2008 at 8:52am | IP Logged Quote mamin123

How much perfomance degration will be if I have about 1000-1500 url mapped such a way ?.

I beleive ISAPI has to check everything in this file and all the content while any request is being made.

 

In fact I do not care for those pages..

Is there any rule which one will do 404 so google know that this page do not exist anymore...I keep this file with 404 redirect for a while and then will remove mapfile after google removes from their list.

 

 

 

 



Edited by mamin123 - 23 September 2008 at 8:54am
Back to Top View mamin123's Profile Search for other posts by mamin123
 
Anton
Moderator Group
Moderator Group


Joined: 30 January 2007
Location: Ukraine
Online Status: Offline
Posts: 4701
Posted: 25 September 2008 at 9:21am | IP Logged Quote Anton

Performance degradation will be absolutely insignificant.

You may try this rule:

RewriteRule ^some-page\.htm$ 404.asp [NC,L]

__________________
Regards,
Anton
Back to Top View Anton's Profile Search for other posts by Anton
 

If you wish to post a reply to this topic you must first login
If you are not already registered you must first register

  Post ReplyPost New Topic
Printable version Printable version

Forum Jump
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot delete your posts in this forum
You cannot edit your posts in this forum
You cannot create polls in this forum
You can vote in polls in this forum