AuthenticateRequest for robots.txt take too long.

ISAPI_Rewrite is Apache mod_rewrite compatible URL rewriter for Microsoft IIS
User avatar
Posts: 33
Joined: 18 Jul 2012, 07:50

AuthenticateRequest for robots.txt take too long.

13 Mar 2013, 07:43

I am have a very strange problem on my IIS 7.5 Server.

As soon as google Robot send a GET request for robots.txt file. ISAPI Module calls AuthenticateRequest method and then it seems that this process never end till the
complete resources i.e 4GB RAM and 100% CPU will be occupied by this process. And I have to restart the server.

PS: initially robots.txt does not exists.

User avatar
Posts: 1264
Joined: 07 Mar 2012, 10:16

Re: AuthenticateRequest for robots.txt take too long.

13 Mar 2013, 11:30

Hello,

Would you, pls, provide us with the rules you're using?

Regards
Andrew

User avatar
Posts: 33
Joined: 18 Jul 2012, 07:50

Re: AuthenticateRequest for robots.txt take too long.

13 Mar 2013, 13:29

Hi,

Here are the rules what I am using

RewriteMap map txt:map.txt

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{QUERY_STRING} (?!admin)
RewriteCond ${map :${lower:$1}|NOT_FOUND} !NOT_FOUND
RewriteRule ^(.*?\.(?:html|gif|jpg|png)) ${map :${lower:$1}} [NC,L,NS,QSA]

User avatar
Posts: 1264
Joined: 07 Mar 2012, 10:16

Re: AuthenticateRequest for robots.txt take too long.

13 Mar 2013, 13:48

This could be not the authentication issue, but permissions issue.
For example ISAPI_Rewrite doesn't have sufficient permissions to read mapfile.
Please, follow this article to resolve it - http://www.helicontech.com/isapi_rewrite/doc/security.htm

Regards
Andrew

User avatar
Posts: 33
Joined: 18 Jul 2012, 07:50

Re: AuthenticateRequest for robots.txt take too long.

13 Mar 2013, 14:19

Hi,

Thank a lot for the prompt response. I don't think that it is a permissions issue. As in general the mapping works.
With the passage of time it consumes more RAM and CPU resources.

But what some how strange is that the some of the following GET requests never come to an end.

Code: Select all
/
/robots.txt

User avatar
Posts: 1264
Joined: 07 Mar 2012, 10:16

Re: AuthenticateRequest for robots.txt take too long.

13 Mar 2013, 14:26

What build are you using?
What IIS version is this?

Would you, please, provide error.log and rewrite.log for a testing request?
Logging issues are described in FAQ - http://www.helicontech.com/forum/10648-FAQ.html

Regards
Andrew

User avatar
Posts: 33
Joined: 18 Jul 2012, 07:50

Re: AuthenticateRequest for robots.txt take too long.

13 Mar 2013, 15:01

We have

Helicon ISAPI Rewrite b Build (3.1.0.78)
IIS 7.5

on Windows Server 2008 R2 Standard Service Pack 1

A screensort is attatched, which shows /robots.txt request is hanged on
Attachments
robotstext_request.png
robotstext_request.png (26.13 KiB) Viewed 3198 times

User avatar
Posts: 1264
Joined: 07 Mar 2012, 10:16

Re: AuthenticateRequest for robots.txt take too long.

14 Mar 2013, 12:03

Hello,

try to uninstall ISAPI_Rewrite and install the latest build. This should fix the problem.
If it doesn't fix the problem, try to completely remove the ISAPI_Rewrite and see GET request are still causing issues. This may not be ISAPI_Rewrite's fault at all.

Regards
Andrew

Return to ISAPI_Rewrite 3.0

Who is online

Users browsing this forum: No registered users and 13 guests