zoo – Helicon Tech Blog

Installing Redmine on Windows in production

Slov — Tue, 03 Jan 2012 15:33:30 +0000

This is a very simple step by step guide to install Redmine on Windows server in production. This guide is applicable to IIS 7+ based servers, which include Windows Server 2008, 2008 R2, Windows 2012, Windows Vista and Windows 7. Latter two are not intended to be used for production solutions. Please follow these steps:

1. Download Microsoft Web Platform Installer, and install it.

2. Run Web Platform Installer and click “Options”.

3. Add Helicon Zoo feed into “Custom feeds” field: http://www.helicontech.com/zoo/feed.xml Select “IIS” as a target web server for production or “IIS Express” for testing.

A new tab named “Zoo” should appear on the main page of Platform Installer.

4. Go to Zoo → Applications and add Redmine, then click install.

This will automatically download and install all required components, including Ruby, Rails, Helicon Zoo Module and Redmine itself.

5. Setup new web-site dialog will appear. Fill it with your server specific values.

6. Go to the /admin/ folder on this web-site. Use login “admin” and password “admin” to enter Redmine Administration panel.

When you install Redmine into a sub-directory additional configuration is required. Please open config/environment.rb file and add the following line at the bottom:

Redmine::Utils::relative_url_root = ENV[ 'APPL_VIRTUAL_PATH' ]

See http://www.redmine.org/projects/redmine/wiki/HowTo_Install_Redmine_in_a_sub-URI for more information.

Custom database

Redmine supports multiple database engines. By default SQLite is installed, however there are example configuration files for MySQL and PostgreSQL within “config” folder of Redmine application. If you wish to use MySQL, for an instance, take “database.yml.mysql” file; name it as “database.yml” and alter according to your MySQL database settings. If you have already run deployment script, then you need it run it again after database change. Rename deploy_done.rb file in the root web site folder to deploy.rb and request some page. Deployment process will be initiated and database migration tasks will be executed.

Sometimes you may need to manually execute tasks against Ruby application. Use Helicon Zoo web console for this purpose. To start web console go to Windows Start –> Programs –> Helicon –> Zoo –> Helicon Zoo Manager select your web site and click Start Web Console button. Then use this console to run for example “rake db:migrate” command and see it’s output in real time.

Manual installation

If you need to install some specific or modified version of Redmine here you can find manual installation instructions that you can adapt for your needs.

First you will need to run Web Platform Installer, go to Zoo –> Templates and install Ruby Project as you normally install application on either IIS or IIS Express.

This will install generic Ruby project running with Zoo and all requirements, including Ruby, DevKit, Zoo Module, etc. After the web site is created you should see Ruby project’s welcome page:

Download Radmine package as a ZIP file here: http://rubyforge.org/frs/?group_id=1850 Unzip content of redmine-x.x.x folder from the Zip file (where config.ru file is located) directly into the web site’s (or application’s) root, i.e. config.ru file will be in the same folder with web.config file.

Now we need to set up database for use with Redmine. You can follow recommendations to create database on this official page: http://www.redmine.org/projects/redmine/wiki/RedmineInstall

The simplest way is to use SQLite as a database because it does not require any database installation. To use it you just need to create a config\database.yml file in the Redmine installation with the following content:

config\database.yml:

# SQLite version 3.x
development:
  adapter: <%= "jdbc" if defined?(JRUBY_PLATFORM) %>sqlite3
  database: db/development.sqlite3
  timeout: 5000

# Warning: The database defined as 'test' will be erased and
# re-generated from your development database when you run 'rake'.
# Do not set this db to the same as development or production.
test:
  adapter: <%= "jdbc" if defined?(JRUBY_PLATFORM) %>sqlite3
  database: db/test.sqlite3
  timeout: 5000

# Warning: The database defined as 'cucumber' will be erased and
# re-generated from your development database when you run 'rake'.
# Do not set this db to the same as development or production.
cucumber:
  adapter: <%= "jdbc" if defined?(JRUBY_PLATFORM) %>sqlite3
  database: db/cucumber.sqlite3
  timeout: 5000

production:
  adapter: <%= "jdbc" if defined?(JRUBY_PLATFORM) %>sqlite3
  database: db/production.sqlite3
  timeout: 5000

Then open Gemfile, find a line with RMagick dependency and comment it out. This is needed because current versions of RMagic does not compile on Windows platform. This gem is optional.

# Optional gem for exporting the gantt to a PNG file, not supported with jruby
platforms :mri, :mingw do
  group :rmagick do
    # RMagick 2 supports ruby 1.9
    # RMagick 1 would be fine for ruby 1.8 but Bundler does not support
    # different requirements for the same gem on different platforms
    # gem "rmagick", ">= 2.0.0"
  end
end

Because first Redmine start may take longer than default 30-second request timeout, open web.config file and add WORKER_REQUEST_TIMEOUT environment variable as shown below:

Or you can use Windows Start –> Programs –> Helicon –> Zoo –> Helicon Zoo Manager application to edit environment variables instead of editing web.config manually:

Rename deploy_sample.rb file from Zoo Ruby project template to deploy.rb:

And request any page on a site – this will initiate deployment process.

After deployment is completed you should see Redmine’s home page. Then proceed with Admin section as form previous chapter.

Offline installation

Sometimes you may need to install Redmine on a machine that is not connected directly to the Internet. Here is the instruction how to prepare offline installation package that you can use to install as many copies of Redmine as you need.

First you will need another machine that is connected to the Internet to prepare Redmine on it. Please follow instructions above and install Redmine using Helicon Zoo. Make sure everything works as you need and all modules and components are installed into web site’s GEM_HOME folder. Helicon Zoo applications are self-contained – this means you can simply move application from one machine to another just by copying web site folder as long as required Zoo Hosting Package (in our case Ruby Hosting Package) is installed on this machine. So we will need to prepare offline package to install Ruby Hosting Package. Fortunately Web Platform Installer provides a command line tool to do this.

On the Internet-enabled machine open command line interface (this could be simple command line or PowerShell) and navigate to Web Platform Installer folder; this is usually C:\Program Files\Microsoft\Web Platform Installer. Then run the following command:

WebpiCmd.exe /offline /Products:RubyHostingPackage /Path:C:\ruby-offline /Feeds:http://www.helicontech.com/zoo/feed.xml

With the /offline key WebpiCmd.exe will download all possible dependencies for the selected product and save downloaded files plus special feed section required for offline installation in the c:\ruby-offline folder. After generation is completed simply copy resulting folder to the Internet-restricted machine. You don’t even need to install Web Platform Installer on this machine because generated folder already contain WebpiCmd.exe file in bin location.

So open command line on Internet-restricted machine and run this command:

WebpiCmd.exe /install /Products:RubyHostingPackage /XML:C:\ruby-offline\feeds\latest\webproductlist.xml /Feeds:C:\ruby-offline\feeds\latest\supplementalfeeds\zooproducts4.xml

This will install Ruby Hosting Package and all possible requirements to run Ruby applications on this server. After installation is completed simply create an empty IIS web site and copy content of Zoo Redmine web site from the Internet-enabled machine and run the site. Please don’t try to make offline copies of Redmine application directly from feed because these packages require Interned access to run ‘gem install’ commands and other deployment tasks. So you will need to copy working and completely tuned Redmine web site instead of trying to install fresh Redmine on Internet-restricted machine.

Node.js on Windows (with performance tests)

Slov — Tue, 06 Dec 2011 15:10:15 +0000

In this article we are going to address basic question which might occur in minds of green web-programmers and those who is going to dive into Node.js learning, namely:

how to install Node.js developer environment of your PC;
how to code;
how to debug;
how to deploy what you’ve done on remote server.

And the performance tests at the end of the article will try answer a reasonable question “Why would I need to learn Node.js.”
So, let’s start…

Node.js is an event-oriented Javascript-based framework for development of web applications. The core concept is that nothing gets blocked during code execution – operations waiting for data transfer, data input, connection establishment or anything else are not present. Everything is based on events, which occur at the moment synchronous operations are waiting for. This sometimes leads to dramatic—dozens of times—performance boost in comparison with old sync systems. With the release of version 0.6.0 of November 2011 Node.js assembly for Windows in announced stable.

Installing working environment

To begin it’s necessary to download and install Web Platform Installer, run it, click Options and put Helicon Zoo Feed link http://www.helicontech.com/zoo/feed/ into “Display additional scenarios” field:

This adds Zoo tab in Web Platform Installer:

Installing Node.js

Under Zoo -> Engines there’s a list of all available web engines, including Node.js. However, we recommend opting for Node.js Package which incorporates not only Node.js itself but also several highly useful modules. So, got to Zoo -> Packages -> Node.js Hosting Package and do Add, Install.

To see all currently supported web frameworks and applications visit Helicon Zoo Gallery. After you agree to the license agreements, it starts downloading and installing IIS (if not yet there), Helicon Zoo Module and node.exe for Windows.

An important system component is Node Package Manager (npm) used for installation of additional modules. Starting from version 0.6.5 of Node.js Node Package Manager declared stable and now is included into Node.js Hosting Package.

Installing WebMatrix templates

Now Node.js is installed and to start doing applications for it it’s reasonable to use WebMatrix templates. These templates simplify creation of blank draft apps which might be used for further development.
To install them follow: Zoo -> Packages -> WebMatrix Templates

If you don’t have WebMatrix – not to worry – it will be downloaded and installed automatically together with installation of templates. After the installation run WebMatrix and choose «Site from Template»:

As you can see, Node.js is not the only framework favoring from WebMatrix templates.

If you follow the URL of the newly created Node.js Site or press Run, you get common «Hello, World!» page.

By default the new site includes framework express for easy web-apps creation. The framework and its dependencies reside in node_modules folder under the site root which is good for deployment of the app on remote server.

The folder “public” is used to store static files. Any file put into this folder will be processed directly by IIS as static, not evoking Node.js. This is especially important to avoid accidental execution of client *.js files on the server.

web.config file contains URL Rewrite rules for static content. Initially every request is checked against the content of public folder (i.e. whether it’s static). This is beneficial for some web apps which mix static and dynamic resources in one folder (often root). If your app is not doing so, you can delete Microsoft URL Rewrite rules from web.config and refer to static files by explicitly specifying public folder.

Additionally, web.config includes configuration directives required for launch of Node.js and Helicon Zoo Module on that site.

Composing the first app

One of the advantages of Node.js is that JavaScript is a popular language widely used in web development. It means you won’t have problems choosing the editor. Free WebMatrix editor is ok to start with.

To illustrate capabilities of asynchronous web frameworks most write a chat. So, the most known demo-app for Node.js is a chat http://chat.nodejs.org/, its source code is available for examination.

We’ve also decided to make a chat – very primitive one – with no users, no sessions, no scrolls and message editing. It can only transmit asynchronous messages to show how long-polling works.

We’ll make use of previously-created Node.js Site. We need to edit server.js and index.html.

Here’s the source code for server.js:

var express = require('express');
var callbacks = [];

// Sends messages to clients
function appendMessage(message){
  var resp = {messages: [message]};
  while (callbacks.length > 0) {
    callbacks.shift()(resp);
  }
}

// Creation of express server
var app = module.exports = express.createServer();
app.use(express.bodyParser());

// Simply respond with index.html
app.get('/', function(req, res){
    res.sendfile('index.html');
});

// Process messages from client
app.post('/send', function(req, res){
  var message = {
    nickname: req.param('nickname', 'Anonymous'),
    text: req.param('text', '')
  };
  appendMessage(message);
  res.json({status: 'ok'});
});

// Wait for new messages
app.get('/recv', function(req, res){
  callbacks.push(function(message){
    res.json(message);
  });
});

// Listen to the port
app.listen(process.env.PORT);

and index.html



Node.js Zoo Chat




Node.js Zoo Chat

    Nickname: 
    Message:

To apply changes press Restart and then Run:

Now we can make sure the chat works by running it in two browsers:

Installing modules

The flexibility and usability of each web framework may probably be defined by availability of extra modules and applicability of third-party technologies. At present Node Package Manager declared stable for Windows, so you ca use it safely for modules installations. NPM is included in Node.js Hosting Package and we have plans to include it into Node.js engine installation itself very soon – just as new Node.js MSI installer will be applicable to use in Helicon Zoo repository. And there’s one trick to remember – by default npm installs modules to node_modules folder under the folder it was invoked from. This is good for further application deployment on remote server as all required modules are included into application folder itself. Thus, to install module for site, go to the site root and execute:

C:\>cd "C:\My Web Sites\Node.js Site"

C:\My Web Sites\Node.js Site>C:\node\npm.cmd install mongodb

npm WARN [email protected] package.json: bugs['web'] should probably be bugs['url']
> [email protected] install C:\My Web Sites\Node.js\node_modules\mongodb
> node install.js
================================================================================
=                                                                              =
=  To install with C++ bson parser do    =
=                                                                              =
================================================================================
[email protected] ./node_modules/mongodb

Another thing worth mentioning is that no all existing modules work for Windows. E.g. the perfect node-sync library won’t work for Windows. This library helps avoid monstrous callbacks paradigm without losing Node.js asynchronous nature, but it relies on node-fibers, which do not work on Windows. Hopes are that in future fibers will be implemented directly into Node.js.

Nevertheless, most modules are fully operational on Windows.

CoffeeScript

If you start working on a more or less complex Node.js project, sooner or later you’ll come to a conclusion that JavaScript is not that friendly. Tons of braces, loads of unnecessary constructions – all that stuff doesn’t add to code readability and makes code management more complicated. Luckily, you are not the first one to notice that, so the problem is already solved. There are many derivative languages based on JavaScript or extending it. Here’s a short list FYI: http://altjs.org/

We take CoffeeScript as the trendiest for now. Code written with CoffeeScript is simple and easy to read. This code is then compiled into original JavaScript and executed. Moreover, JavaScript code may be converted into CoffeeScript. For example, server.js script from our chat in CoffeeScript becomes this:

express = require("express")
callbacks = []

// Sends messages to clients
appendMessage = (message) ->
  resp = messages: [ message ]
  callbacks.shift() resp  while callbacks.length > 0

// Creation of express server
app = module.exports = express.createServer()
app.use express.bodyParser()

// Simply respond with index.html
app.get "/", (req, res) ->
  res.sendfile "index.html"

// Process messages from client
app.post "/send", (req, res) ->
  message =
    nickname: req.param("nickname", "Anonymous")
    text: req.param("text", "")

  appendMessage message
  res.json status: "ok"

// Wait for new messages
app.get "/recv", (req, res) ->
  callbacks.push (message) ->
    res.json message

// Listen to the port
app.listen process.env.PORT

Learn more about CoffeeScript: http://jashkenas.github.com/coffee-script/
To install CoffeeScript run: C:\node\npm.cmd install coffe-script

Debugging Node.js-apps

There’s a good tool for Node.js apps debugging — node-inspector. It is already included into node_modules folder in template of Node.js-site. node-inspector works as follows:

application to be debugged in run in debug-mode (execute node.exe with –debug or –debug-brk parameter);
node-inspector, being also a web-app, is launched;
these 2 apps communicate via WebSocket protocol to share debug information;
page with the application being debugged is opened in browser;
webkit-compatible browser opens node-inspector interface with debug info; its interface resembles that of Web Inspector in Google Chrome or Safari.

In root folder of template-based node.js-site there’s start_debug.cmd file which runs debugging for current app and opens pages in the browser for debugging.

This is how debugger looks in the browser:

Deploying on server

Now we have an app and want to put in online. What we need is a server, and now putting together Windows server and Node.js is easy as never before. We only need to go through several steps we did at the beginning of the article (which we used to deploy working environment). Namely: install Microsoft Web Platform Installer, add Helicon Zoo feed into it and set up Node.js Hosting Package from Zoo repository. Done – server is ready to run our app. Server platforms supported include Windows 2008 and 2008 R2, x86 and x64.

What needs to be done next is create blank web-site on the server using IIS or hosting panel (if we are creating our own hosting) and copy the app into the site with FTP or WebDeploy. In case of WebDeploy all necessary permissions for the folders will be assigned. One can also use Git or any other version control system, but that goes beyond the scope of this article.

Initially, Helicon Zoo Module was being developed for configuration of hosting solutions. And all applications within Zoo are separate and do not interfere. The module itself with default settings operates in automatic mode creating one worker process, when the load is low, or bearing new workers (up to the number of processor cores) to ensure maximum performance when the load goes up.

Helicon Zoo adopts the concept of engines and applications. Engines define what to run and how, using which protocol and port, min and max number of workers allowed and other general settings, which are specified globally in applicationHost.config. Then under the site you can create an application relying on a particular engine and pass all parameters required for its flawless operation. This helps isolate hosting administrator work from clients and clients from each other.

Node.js performance testing

Testing server characteristics: Core 2 Quad 2.4 GHz, 8 Gb RAM, 1Gb LAN. For load generation we used more powerful machine and Apache Benchmark command «ab.exe -n 100000 -c 100 –k». To test Apache and Nginx we took Ubuntu 11.04 Server x64. IIS 7 tests ran on Windows Server 2008 R2. Nothing virtual — 100% hardware.

We’ve conducted 3 tests. In the first one Node.js was supposed to output current time in high resolution. Time was chosen to be sure responses didn’t come from cache. Second test implied reading from MySQL database, third one — writing to DB.

Here are the results (values on the graph mean requests per second):

Impressive, isn’t it? Time to explain what these tests measure. It’s not entirely correct to call them performance tests as we do not compare different processors. While processor may have performance, web server measurement unit is quite the contrary — how much processor time it takes to process one request.

The first test measures raw expenses on request processing for each web-server and their ability to use processor resources. There’s no way this or that set of technologies can respond faster on this processor. Nginx on Windows was far behind on this test because on this system Nginx opens new connection to back-end upon each request. While Apache on Windows surprised with connection pooling and true threads.

The second and third test show how web-server expenses grow with the “weight” of request. However, they are influenced by a number of other factors, such as performance of file system, DB drivers and the DB itself. Out of curiosity we’ve tested the joint Windows + Zoo + MongoDB to see the difference with MySQL. And it gave 6793 rps for reading and 2906 rps for writing. Writing speed is truly amazing.

Another interesting fact is that software and hardware used in these tests is absolutely the same as in Django tests in this article. So their results may be compared. Undoubtedly Node.js scripts are much more light-weight, we didn’t use templates, ORM etc,, but anyway it’s worth thinking about.

Responding to readers’ requests we are posting detailed ab graphs. We’ve re-done the first test with simple time output, ‘case it most verbosely depicts web server operation. Config files and js scripts being tested may be downloaded from here. There are only includes, everything else left by default. Horizontal axis shows requests, vertical one — response time in milliseconds.

Windows, IIS7 + Zoo, “time output”:

Ubuntu, Apache, “time output”:

Ubuntu, Nginx, “time output”:

Summing up

I believe that Node.js is a rather promising trend. It boasts great performance and flexibility. What is especially pleasing is that it’s equally good on both Unix and Windows and uses relevant technological solutions for each system which the tests vividly prove.

Support of Erlang and Java in Helicon Zoo is on the way. It will be of interest to compare performance of these technologies as well. For now Node.js is an unquestionable leader in performance among supported frameworks.

Running Django on Windows (with performance tests)

Slov — Tue, 20 Sep 2011 14:39:36 +0000

Django is a trendy Python-based framework for web applications. Its ever-growing popularity owes to availability of quick development tools, inbuilt administrative interface and high operation speed. Unfortunately, until now there was no easy tried-and-true way to deploy and run Django-apps on Microsoft IIS web-servers. Here we introduce Helicon Zoo – a solution to run popular web-frameworks and applications with Microsoft IIS. It bases on Microsoft Web Platform Installer (WebPI) technology to deploy apps. WebPI also handles different dependencies and takes care of installation of necessary components, such as Python, IIS, various database drivers and modules. And, of course, Helicon Zoo itself which sticks it all to Microsoft IIS 7+.

Development environment configuration

Web development process implies use of two relatively independent environments – development and production. Helicon Zoo may be used in production as well as on developer’s machine, or in both places. In either case the sequence of actions might be:

To start, you need to download Web Platform Installer from Microsoft website (http://www.microsoft.com/web/downloads/platform.aspx) and install it. WebPI already includes wide range of frameworks and applications for IIS like PHP, ASP.NET, WordPress, Drupal, phpBB. To launch Helicon Zoo add new feed to WebPI:

Run WebPI and click Options. In Custom Feeds box put http://www.helicontech.com/zoo/feed.xml and click Add feed:

Please note web server choice – IIS Express or IIS. The main difference is that IIS Express runs as interactive user application which usually means Administrative permissions if you are logged in as Administrator. This simplify development process and decrease number of possible issues you may encounter with insufficient NTFS permissions to run application. With IIS web applications are executed as a restricted user and may require additional permissions tuning, but you will get environment that is more close to those which will be used to run application in production. In this article we will be using IIS Express with WebMatrix as a development environment and IIS for production.

After adding custom feed new Zoo tab will appear with Applications, Templates, Packages, Modules and Engines sections in it.

Applications – are some ready to use web applications that are included into Helicon Zoo feed, that come with all required dependencies and can be installed on a clean system in a few clicks. You can use this section if you simply need to install one of these applications on your server.
Templates section contain empty projects for each web application technology that is currently supported by Helicon Zoo. You use these templates as a starting point to develop application of particular technology or to host existing application. For example in this article we are going to use Python project template.
Packages include complete Hosting Packages for each web application technology that is supported by Zoo. You install one or more hosting package from this section to enable hosting of the whole technology class on the server. Each package contains all dependencies that may be needed. For example Python Hosting Package contain Python 2.7, Virtualenv, Pip, Flup, Python Imaging Library, MySQL Driver for Python 2.7, Twisted, Helicon Zoo Module and some other modules that may be needed and that have to be installed globally. We will be installing Python Hosting Package to configure production environment later.
Modules contains some useful applications and modules that are usually installed on server globally, like Helicon Ape, MongoDB, PostgreSQL, RabbitMQ, etc. Some of these modules are not intended to be installed separately from Hosting Packages.
Engines section contains engines itself, like Ruby, Python, Perl. Please note that installing these engines will not generally enable them in IIS environment. You should not install these engines manually, please use corresponding Hosting Package or Template instead.

Creating Django application

The best thing about Helicon Zoo and Web Platform Installer is that creating new application and installing application environment is all done in one step because WebPI will check and install all needed dependencies automatically. Please go to Zoo –> Templates, choose Python project and install it.

Depending on the server chosen (IIS or IIS Express) and your system configuration many components may be downloaded and installed first time.

Warning: If you have already manually configured Python environment, please use packages that come with Helicon Zoo through the feed and Web Platform Installer instead. Even though it is possible in theory to use your custom Python installation with Helicon Zoo, we highly recommend to use packages from Zoo feed. If you ignore this requirement, troubleshooting your installation may be complicated.

If you have chosen IIS Express as a web server, after installation is finished WebMatrix will open automatically and you should see project’s index page with further instructions after short deployment process. With IIS installations additional step is required where you choose port and host name bindings, a folder on disk where to put web site, etc. IIS Express web sites are created in \Documents\My Web Sites\. During the deployment process special script, that will be explained later, creates a new Python virtualenv inside the folder of web site. This virtualenv should be used for all further installations of modules and components and Helicon Zoo will start application using this virtualenv instead of global Python settings.

Initial project content is rather simple:

An empty console folder that is simple placeholder if you wish to configure authentication for web console access later; static folder should be used to store static files (including Zoo welcome page); venv folder contains Python virtualenv; deploy_done.rb file will be explained later; requirements.txt file will contain project dependencies and web.config file that is essential to configure Zoo application in this folder. The virtualenv configured in this folder is a normal Python vritualenv, with the only difference that Zoo will not call python.exe file from this folder by default. With typical settings Zoo Python 2.7 engine will call python.exe file from the Python installation folder (i.e. C:\Python27), configuring all other variables to point to the virtualenv inside web site folder.

The instruction on Welcome page asks us to click on a link and start Web console, but we will learn one more tool before – Helicon Zoo Manager. Go to Start menu, Helicon –> Zoo –> Helicon Zoo Manager (either for IIS or IIS Express depending on the server type you are using). This manager lists all your current web sites and here you can modify configuration of Helicon Zoo Module, enable or disable engines, set environment variables, start Web Console or IDE, etc. By default Helicon Zoo will run Python application using Python 2.7 WSGI engine and you can change this using Helicon Zoo Manager. So, let’s click Start web console button to launch console for this application:

Then type the following command to download and install Django into the virtualenv folder:

pip install django

You may ask why using this web console if I can just run cmd.exe or any other IDE to run commands? The answer is because Helicon Zoo web console is designed to run commands in the isolated environment of your application, so all these commands are applied to the application you are working with, using local folders and environment variables and executed by the same interpreter and same IIS Application pool user that runs the application itself. This is needed to keep applications portable because all modules and components are installed into the application folder and execution environment is easily replicable by installing Helicon Zoo Hosting Packages on other machines. On the other hand if you launch Windows console from start menu you may actually have number of environments and interpreters installed on your machine, like several different versions of Python. With Windows console when you run a command you can’t tell for sure what exact version of interpreter you are calling, where it is located, where will it store it’s settings, etc. IDEs and commands to install modules will usually install them globally into the system, so your application will lose portability. There could be conflicts between different versions of engines or modules installed in the system when you run global command line interface. This is why it is always recommended to install web application engines like Python or Ruby distributions using Helicon Zoo repository, for example by installing Hosting Packages, instead of downloading and installing engines manually. And use of Helicon Zoo web console or launching IDE from Helicon Zoo Manager may also be essential if you want to avoid version conflicts and retain application portability.

This web console executed by Zoo Module as HTTP application in your browser. Anonymous remote requests to console are prohibited by Zoo engine for security reasons. So if you wish to access console on a remote server you will have to enable one of the authentication methods for the console folder (or whatever location you have configured as a console). Or you can use Windows IIS Manager to connect to remote server and start console from Helicon Zoo IIS Manager snap-in. The Helicon Zoo Manager installs snap-in for Internet Information Services Manager, which you can use even in remote mode, when IIS Manager is connected to a remote server.

The one-time hash code will be used to authenticate console session and will be invalidated every time when you close the console window. The ability to start web console can be enabled or disabled globally and for individual applications using Helicon Manager, which is useful for hosters. Please read more about web console here.

After Django installation is finished type the following command in console to create an empty Django project named ‘project’:

django-admin.py startproject project

Then as per instruction add the following environment variable using Helicon Zoo Manager: DJANGO_SETTINGS_MODULE=project.settings

Now, if you refresh web site default page you should see Django project welcome page:

Integrated Development Environment

Another useful feature of Helicon Zoo Manager is the ability to start IDE for the application environment. This is not just a shortcut to your favorite IDE. Before launching IDE, Zoo Manager will configure environment according to the environment variables of the selected application. Most current IDEs can read these environment variables to configure locations correctly. Locations like virtualenv folder, working directories, Path variable with correct locations of Python interpreter of required version, etc. Open Helicon Zoo Manager and click on Start IDE. When you do this first time for the application a small Select IDE dialog will appear. By default it opens Windows Command Line (cmd.exe) and this is a convenient replacement for the Web Console we’ve used in previous chapter and if you develop application locally. This command line interface will be launched with all path configured for your application, therefore ‘pip install’ command will install modules into the application folder same as with web console. The difference with web console is here cmd.exe is executed as interactively logged on user, while web console is executed as IIS application pool user, which may differ in permissions significantly. So for development purposes and on local machine using Start IDE command is even more convenient than the web console.

But instead of using ascetic command line you can configure your favorite IDE to start with this command. Environment variables will be configured before launching application and IDE will know correct locations of files, like GEM_HOME, location of Ruby interpreter, etc.

Below please find list of popular Python IDE’s that run on Windows:

Pydev (freeware);
Python Tools for Visual Studio (freeware, requires Visual Studio);
PyCharm (commercial);
Komodo (commercial);
Wingware (commercial).

Installing native modules

Besides modules written on Python, there are some modules that needs to be compiled during installation procedure. These are so-called native modules. Normally compilation process is automatic and requires only presence of C++ compiler in the system. No write access to system folders required as compiler will save all output files into the module installation directory. The tricky thing here is the version of C++ compiler should be the same as version of the compiler used to build Python distribution itself. Python 2.7 package that is currently provided with Helicon Zoo has been built using Microsoft Visual Studio 2008 (v. 9.0). So to provide support for native modules installations you only need to install this version of Visual Studio in the system. There is a freeware version of this studio available from Microsoft which you can download here: Visual Studio 2008 Express.

After installation you need to restart Windows so the Python starts using this compiler to build native modules. If you have several versions of C++ compiler installed in the system, sometimes it is necessary to specify the exact version that will be used by Python. For this purpose you can set the following environment variable. Simply add it to the Python engines using Helicon Zoo Manager:

VS90COMNTOOLS=C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\Tools\

Developing an application

Now, let’s create a simple Django application as a concept proof. For a convenience on development machine I suggest you to set up file changes watch mask. Since Python engine does not reload modified files automatically, this will restart Python every time when any *.py file is modified in web site folder. Please add the following environment variable to web application: WATCH_FILE_CHANGES_MASK=*.py

We will follow Tutorial #1 from Django documentation. We’ll skip chapters explaining Python server configuration as we already have it running and go to Database setup chapter. We are going to use SQLite3 as a test database. So, open project\project\settings.py file in the web site folder and modify DATABASES section as follows:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3', # Add 'postgresql_psycopg2', 'mysql', 'sqlite3' or 'oracle'.
        'NAME': 'DEV_DB.sqlite3',              # Or path to database file if using sqlite3.
        # The following settings are not used with sqlite3:
        'USER': '',
        'PASSWORD': '',
        'HOST': '',                      # Empty for localhost through domain sockets or '127.0.0.1' for localhost through TCP.
        'PORT': '',                      # Set to empty string for default.
    }
}

Then start IDE or cmd.exe using Helicon Zoo Manager. We will need a command line interface. First move to the ‘project’ folder:

cd project

Run sincdb command to create database structures:

python manage.py syncdb

And create ‘polls’ application inside the project:

python manage.py startapp polls

Modify project\polls\models.py as follows:

from django.db import models

class Poll(models.Model):
    question = models.CharField(max_length=200)
    pub_date = models.DateTimeField('date published')

class Choice(models.Model):
    poll = models.ForeignKey(Poll)
    choice_text = models.CharField(max_length=200)
    votes = models.IntegerField(default=0)

Edit project\project\settings.py again and add ‘polls’ to INSTALLED_APPS section:

INSTALLED_APPS = (
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'polls',
)

And run syncdb again:

python manage.py syncdb

Edit polls\views.py:

from django.http import HttpResponse

def index(request):
    return HttpResponse("Hello, world. You're at the poll index.")

Add URL route by creating file polls\urls.py with following content:

from django.conf.urls import patterns, url

from polls import views

urlpatterns = patterns('',
    url(r'^$', views.index, name='index')
)

Now let’s connect this route to the main urls.py by editing project\urls.py as follows:

from django.conf.urls import patterns, include, url

# Uncomment the next two lines to enable the admin:
# from django.contrib import admin
# admin.autodiscover()

urlpatterns = patterns('',
    # Examples:
    # url(r'^$', 'project.views.home', name='home'),
    # url(r'^project/', include('project.foo.urls')),

    # Uncomment the admin/doc line below to enable admin documentation:
    # url(r'^admin/doc/', include('django.contrib.admindocs.urls')),

    # Uncomment the next line to enable the admin:
    # url(r'^admin/', include(admin.site.urls)),
   url(r'^/?', include('polls.urls')),
   )

Now start browser with application’s home page. You should see the “Hello world” response of our application:

We will not guide you through the rest of the Django tutorial as purpose of this article was to prove the concept and show how to configure environments using Helicon Zoo. You can keep up with Django tutorial here.

Deploying application to production server or hosting

So, we’ve created the application and now, logically, want to publish it to the Internet. For that purpose we’ll set up Windows server to work with Python in production. We’ll have to repeat several steps from the beginning of the article which were used to install development environment, but this time we’ll configure production IIS server. If you are considering to organize Python hosting on Windows servers you will need to complete exactly the same steps as following:

Install Microsoft Web Platform Installer.
Add Helicon Zoo feed. Naturally this time you choose IIS as a target server instead of IIS Express in WebPI options.

Now install Python Hosting Package from Zoo –> Packages.

This will install Python 2.7 normally into C:\Python27 folder, Virtualenv, Pip, Flup, Python Imaging Library, MySQL Driver for Python 2.7 and Twisted. All these components are needed for normal Python applications operation, rest of the components will be installed into the application folder itself. Installing Python Hosting Package will also set correct NTFS permissions for Python installation folders for default IIS Application pool settings. You can update Python installation packages manually then, leaving installation folders as they are, because these folders are hardcoded in some WebPI components. But remember – installation packages form Zoo repository has been tested with Zoo while you are upgrading on your own risk. Python Hosting Package may also install IIS and other Windows components as dependency if they are not already installed in the system. Eventually it will install Helicon Zoo Module which is required to host these applications with IIS.

After these steps are completed the server is ready to host our application. For the moment the following server platforms are supported: Windows Server 2008 and 2008 R2 and Windows 2012, all 32-and 64-bit versions when applicable. The reason why older versions are unavailable is because Helicon Zoo Module uses native IIS 7 API, therefore everything prior IIS version 7 is unsupported and all newer versions of IIS should be fine.

So, at first, we create an empty web-site via IIS manager or your hosting panel. Then simply upload entire web site folder with your application to the server via FTP or Web Deploy or any other way. I would recommend to configure Web Deploy on the production server. This tool makes deployment of applications from WebMatrix or Visual Studio really easy, plus all application folders and files will be given proper permissions automatically, as they’ve been set by Helicon’s Python project template. Generally you may need to enable write permissions for entire web site folder for the user running application because Python application will want to write things sometimes. You can also use Git or any other version control system and deployment system, but that falls beyond the scope of this article, same as write permissions fine tuning.

Then, in general, you just navigate to the web site and it opens. The application will be executed on Windows Server under IIS with the help of Helicon Zoo Module. This module was initially designed as a hosting solution, so all applications are isolated and do not interfere. The module with its default options works in fully automatic mode, creating one worker process per application by default, when the load is low and increasing the number of workers up to the number of CPU cores of the server providing maximal performance under heavy loads. These settings can be changed on per-engine level using Helicon Zoo Manager.

Helicon Zoo implements the concept of engines and applications. The engines define how to run an application, which interpreter to use, which protocol and port, what maximum number of workers are allowed and other global settings which are defined in applicationHost.config, so if the user has no write permissions to applicationHost.config then he can’t change engine settings. Then, Helicon Zoo application ‘uses’ the engine by referencing it from web.config file from inside web site folder. Each engine may have list of parameters – Environment Variables, which users can set in web.config files located in application folders. This concept allows for separation of hosting administrator duties from the clients’ burden (and client from client as well). You can learn more about Helicon Zoo Module configuration here.

Sometimes, when your application is not just an empty Hello World as in our example, simply copying files to the production environment is not enough. For example, your application may use external database server and you may need to execute database migration tasks in production environment before the new code could be executed. For this purpose Helicon Zoo offers very convenient tool called deploy scripts. Please notice DEPLOY_FILE=”deploy.py” environment variable in the Helicon Python project template. This variable means that every time when Helicon Zoo engine finds deploy.py file in the root of the web site it will do the following:

All currently running Python processes for this application will be shut down using soft-shutdown sequence.
All running requests and transactions will be allowed to run till they are finished, unless soft-shutdown timeout is exceeded.
All newly coming HTTP requests to the application will receive 503 Application Maintenance status.
These new requests will see the “Application deployment in progress” page.
When no running workers left, the deploy.py file will be executed in a separate process using same interpreter (i.e. Python 2.7) and same user as declared in web site’s app pool.
After deploy script execution is finished with no errors all users currently seeing “Application maintenance” message will be redirected to the pages they where originally requested.
The deploy.py file is renamed to deploy_done.py.
New worker processes will be started and application execution continues.

Here is how Application deployment in progress message looks by default:

In Python project template there is already deploy.py file. You may have noticed it’s execution on the first application start. During this first execution this deploy script checks for presence of virtualenv in the folder and configures a new virtualenv if no existing found. Additionally this file contains common deployment instructions for database migrations, modules installation using requirements.txt file, etc. So if you exclude /venv/ folder when uploading your application to the production server, it will be recreated during next deployment process and all requirements will be installed.

To initiate deployment process you will only need to rename the deploy_done.py file into deploy.py and push it to server. You may want to make it your last change to the project when you upload changes so to make sure all other scripts and files has been updated before deployment process initiates. If the WATCH_FILE_CHANGES_MASK environment variable is not set, Python will not load updated code files unless restarted, so initiating deployment process will do all things synchronously – migrate database, install requirements and then load new code into engines. This is so-called “cold application maintenance” which is needed sometimes because if other user requests (with either new or old code) will be running while database migrations and other deployment tasks are executed the data could be corrupted or user could get unpredicted responses. Helicon Zoo minimizes application downtime to even seconds and automates whole deployment process so it becomes easy to deploy large applications across array of servers using simple techniques.

Please read more about deployment scripts in Helicon Zoo documentation.

Performance

And now performance test for dessert. Testing machine acting as server was Core 2 Quad 2.4 GHz, 8 Gb RAM, Gigabit LAN. To generate load we used more powerful PC with Apache Benchmark. To measure Apache and Nginx performance Ubunthu 11.04 Server x64 was used. IIS 7 tests ran on Windows Server 2008 R2. No virtual machines – only bare hardware. As transport on Nginx we used the most advanced uwsgi, as well as wsgi and fast_cgi for comparison. On IIS 7 we’ve also compared with PyISAPIе.

There where two Django scripts created as testing pages. The first one outputs current time in high resolution; this is done to ensure pages are not taken from cache. The second one does the same but previously saves the result into database. It’s all done using templates in order to apply real Django infrastructure; DB used is MySQL. All settings were left default as the task was to test the most common configurations. Here are results (in requests per second):

No surprises here as Python performance on Windows may be slower than Ubunthu version. Taking this into consideration Helicon Zoo transport performance should be really high. Uwsgi is ahead probably due to closer integration with Django.

The results for the second script are not that smooth. Why Nginx + fcgi + MySQL showed only 175 requests per second remains unknown. MySQL on Windows score is also frustrating, although on shared hosting the problem might not be that critical. The thing is that performance drops due to internal MySQL locks, while the server is not even loaded for 20% while generating these 104 requests per second. It’s reasonable to assume that by increasing number of sites on the server and consequently the number of DBs, if they do not interlock with each other, the total server performance will be acceptable.

Thus we decided to add MS SQL Express into the tests. The result was easy to explain with Python and its database driver being the bottleneck, though in general the picture is quite promising. Unfortunately PyISAPIe was unable to work with MS SQL Express and was excluded from tests.

It is worth to mention the ability of IIS 7 to handle great number of connections. IIS 7 + Helicon Zoo easily held thousands of concurrent connections, we simply didn’t have testing powers to generate enough connections to trigger any problems in this test. Ubuntu with default settings started throwing connection failures when the number of connections increased. Moreover, Apache appeared to be greedy for memory. During the test with the number of connections going up Apache swallowed about 3 GB in 20 seconds.

Running Django, Rails and Mojolicious on Windows Server 2008 with IIS 7

ruslan — Fri, 24 Jun 2011 14:06:17 +0000

Recently released Helicon Zoo is a repository of popular web frameworks for Microsoft IIS. It makes installing and running Django, Ruby on Rails and Mojolicious on Windows Servers a piece of cake.

THIS ARTICLE IS OUTDATED

Please read following articles to find information how to run Django, Rails and other popular web engines on IIS:

And you can find usage examples for various web engines on a usage page.

How it works

Helicon Zoo utilizes Microsoft Web Platform Installer (WebPI) to make installation of web frameworks and all required dependencies easy and smooth. At first, you need to download and install WebPI from Microsoft web site here. This is a great tool that is already packed with many ASP.NET and PHP applications and dependencies to install a whole technology stack on your server. And the greatest thing about WebPI is that it can be extended by adding custom repositories. So copy this link – http://www.helicontech.com/zoo/feed/, open WebPI options

put it into the Display additional scenarios field and click Add feed.

Now if you select Applications, Tools at the bottom of the list, you will find new applications: Blank Django Project, Blank Rails Project, Blank Perl Project, Blank Mojolicious Project

These are all empty “Hello World!” applications used to install all required dependencies which you can stuff with your code later.

So let’s choose Blank Django Project, click Add and Install. This will create new IIS application with pre-defined configuration. After download and installation are completed you can use Launch application in browser link:

What’s inside

The core of Helicon Zoo is a native IIS 7 (or IIS Express) module that provides FastCGI support. This module differs from Microsoft FCGI module generally developed to provide PHP support for IIS. Helicon Zoo Module is built using super capacitive asynchronous I/O Completion Port technology. It supports Pipe and TCP transports and additional level of configuration to run many different FastCGI-enabled frameworks. These includes flup tor Python, Rack for Ruby on Rails, FCGI and Plack for Perl and Mojolicious.

The core configuration of Helicon Zoo Module is located in the section of applicationHost.config file. This section defines all FastCGI engines that will work with Zoo. Here is an example of Django FastCGI engine definition using pipe transport; zoofcgi.py is our Python worker with pipe transport support:

How to deploy Django projects

Helicon Zoo offers simple way to deploy your Django applications on the server or test machine. First you install Blank Django Project in desired location using Web Platform Installer, then you overwrite this project content with your project files and set up deployment into this forlder. Blank Django Project installs all required dependencies on target machine and puts web.config (the only file needed for your application) in the root directory of application.

Web.config

Configuration of Zoo-powered applications is done with web.config file. Here is an example of such file to configure Django application for 32-bit.

Several important things are defined here:

scriptProcess="python.2.7.pipe" — a reference to FastCGI engine defined in the applicationHost.config;
environment variable PYTHONPATH — a path where Python will check to load modules. Here path to this IIS application is appended to allow Python to load additional modules directly from the directory of your application without installing them globally in the system.
environment variable DJANGO_SETTINGS_MODULE — a Python path to the configuration file settings.py of your Django project, for example ‘mysite.settings’.

Static content

It is recommended to serve static content omitting Django to save processor time and increase performance. You can easily do this by putting all static files into a directory with the following web.config in it. This will disable Django module in directory:

Django configuration example

Let’s assume our Django project consists of three applications:

site — a site itself with pages templates, static files, urls.py and settings.py;
blog;
store.

Static files are located in site/media and are available as /media/ virtual directory (i.e. MEDIA_URL='/media/').

After we install Blank Django Project in the root of web site and copy our Django application project in it we will have the following directory structure:

/media/ — is a virtual directory which is mapped to site/media and Django is disabled there using web.config file example above. In the root web.config PYTHONPATH points to the site root directory and DJANGO_SETTINGS_MODULE is set to site.settings:

That’s all!
Sincerely, Helicon Tech team.