maxInstances and number of cores

[kerin Profile]

Hi all,

Is the possible range of values for maxInstances in any way tied to the number of cores on the host?

I'm currently using HeliconZoo on Azure small instance WebRoles, which have a single core, and it seems like the server's resources aren't being fully utilised - during load testing response time quickly climbs from < 1 sec to ~ 20 sec, but the server's CPU utilisation peaks at around 10% with very little RAM used, to it seems like increasing the number of workers is the obvious thing to do. Attempting to do so results in a somewhat vague error message from Azure:

'Role instances recycled for a certain amount of times during an update or upgrade operation. This indicates that the new version of your service or the configuration settings you provided when configuring the service prevent role instances from running. The most likely reason for this is that your code throws an unhandled exception. Please consider fixing your service or changing your configuration settings so that role instances do not throw unhandled exceptions. Then start another update or upgrade operation. Until you start another update or upgrade operation, Windows Azure will continue trying to update your service to the new version or configuration you provided'

This may be because HeliconZoo doesn't support having more worker instances than cores, or it may be an error in my startup task which sets the number of instances (after HeliconZoo has been installed):

Code: Select all

%windir%\system32\inetsrv\appcmd set config -section:system.webServer/heliconZooServer -[name='python.2.7.wsgi'].maxInstances:10

Any info on this much appreciated.


Thanks,

Kerin

[kerin Profile]

Turns out this was in fact an error in my startup task, which should've been:

Code: Select all

%windir%\system32\inetsrv\appcmd set config -section:system.webServer/heliconZooServer -engines.[name='python.2.7.wsgi'].maxInstances:10

Increasing maxInstances doesn't seem to have had any noticeable impact on concurrency however - is there a rule of thumb for how many instances a single-core server can comfortably run?

[Yaroslav]

You can also set minInstances parameter for the engine you are using - this will tell Zoo to pre-launch required number of workers and keep it running.
If you are hitting only 10% CPU and engines are not created automatically this is possible number of engines is not a bottleneck you are hitting.
What is the application you are using and what engine it runs on? And please make sure you are running latest version of Helicon Zoo Module form the feed.

[kerin Profile]

This is a custom django application with the python.2.7.wsgi engine, although we've tried python.2.7.pipe as well with the same results. We are using the most recent version of the Helicon module, as our Azure deployment process runs the webPI installer on each deployment.

Our app doesn't use a database or disk access so should run entirely in memory, which is why we're surprised to see such poor performance:

$ ab -n200 -c25 <host>

Completed 100 requests
Completed 200 requests
Finished 200 requests

Server Software: Microsoft-IIS/7.5
Server Hostname: <host>
Server Port: 80

Document Path: /
Document Length: 31729 bytes

Concurrency Level: 25
Time taken for tests: 87.241 seconds
Complete requests: 200
Failed requests: 0
Write errors: 0
Total transferred: 6455400 bytes
HTML transferred: 6417048 bytes
Requests per second: 2.29 [#/sec] (mean)
Time per request: 10905.129 [ms] (mean)
Time per request: 436.205 [ms] (mean, across all concurrent requests)
Transfer rate: 72.26 [Kbytes/sec] received

It may well be that the bottleneck lies with IIS rather than the Helicon module, but I'm not that familiar with IIS so I'm not sure where the best place to start is.

[Yaroslav]

Do you run ab locally or remotely? If you run your tests remotely you are probably testing network connection speed to your instance.
Also there could be some bandwidth throttling or connection speed limitation configured on the instance or IIS itself? Low CPU utilization and single worker process may indicate slow client connection.