Migrating Topincs to nginx

Rafa_F · July 5, 2013, 3:14pm

Dear list,

This is going to be a long post, but I hope you will bear with me. I
am trying to find a way to run Topincs, an agile web application
framework based on Topic Maps paradigm, on nginx. Topincs is a
typical LAMP app, so I hoped I would be able to figure it out on my
own, but unfortunately not. Even if you find this post too long and
boring to read, I would be grateful for pointing me to resources
concerning migration from apache2 to nginx and descriptions of
successful cases.

First a description of the situation by Topincs’ author:

"My main objective in the Apache and Topincs integration was that it
is possible to occupy any URL space below a domain without the need
for a general ‘root’, e.g. everything under /topincs. Plus it should
be possible to run different stores under different Topincs versions,
just in case.

So Apache does two things for Topincs:

The rewrite rules basically cut off the store path prefix (e.g.
/trial/movies) and pass the result on to
TOPINCS_HOME/docroot/.topincs.
Based on the path prefix it maps the URL to a store (database).
This is done by setting an Apache environment variable in the
configuration. This variable is read in .topincs only.

So once you manage to provide .topincs with the above, you are set." (
http://tech.groups.yahoo.com/group/topincs/message/673 )

So let’s start with a working Apache configuration. I’m running Debian
Wheezy, uname -a gives:

Linux box 3.2.0-4-amd64 #1 SMP Debian 3.2.46-1 x86_64 GNU/Linux

/etc/apache2/sites-enabled/000-default contains:

<VirtualHost *:80>
ServerAdmin webmaster@localhost
DocumentRoot /var/www/

Options FollowSymLinks
AllowOverride None

<Directory /var/www/ >
Options Indexes FollowSymLinks MultiViews
AllowOverride all
Order allow,deny
allow from all

ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory “/usr/lib/cgi-bin”>
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all

ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
Include “/home/apollo/topincs/conf/httpd.conf”

/home/apollo/topincs/conf/httpd.conf :

RewriteEngine on
<Directory “/home/apollo/topincs/docroot”>
Order allow,deny
Allow from all
DirectoryIndex index.php
AddType ‘text/html; charset=UTF-8’ .html
DefaultType application/x-httpd-php
php_value include_path
“/home/apollo/topincs/php:/home/apollo/topincs/vendor/php”
php_value default_charset “UTF-8”
php_value magic_quotes_gpc “0”
php_value max_execution_time “7200”
php_value memory_limit “500M”
php_value short_open_tag “0”

<Directory ~
“/home/apollo/topincs/docroot/[0-9].[0-9]+.0-9?”>
FileETag none
Header set Expires “Fri, 31 Dec 2020 23:59:59 GMT”
Header set Cache-control “public”

Include “/home/apollo/topincs/conf/*.httpd.conf”

/home/apollo/topincs/conf/ contains mercury.httpd.conf:

<LocationMatch ^/mercury/.>
SetEnv TOPINCS_STORE mercury

RewriteRule
^/mercury/([3-9].[0-9].[0-9]./(.core-topics|css|images|js|vendor|fonts).)$
/mercury/$1 [PT,E=TOPINCS_STORE:mercury]
RewriteRule ^/mercury((.|/).)$ /mercury/.topincs?request=$1
[PT,L,QSA,E=TOPINCS_STORE:mercury]
Alias /mercury “/home/apollo/topincs/docroot”

The (useless) nginx setup I came up with so far is following:

/etc/nginx/nginx.conf

user www-data;
worker_processes 4;
pid /var/run/nginx.pid;
events {
worker_connections 768;
}
http {
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log debug;
gzip on;
gzip_disable “msie6”;
include /etc/nginx/conf.d/.conf;
include /etc/nginx/sites-enabled/;
include fastcgi_params;
}

/etc/nginx/sites-enabled/topincs :

server {
listen 80;
root /home/apollo/topincs/docroot;
index index.php;
location / {
fastcgi_pass localhost:9000;
fastcgi_index “/index.php”;
allow all;
try_files $uri $uri/ /index.php;
}
location ~ .php$ {
include fastcgi_params;
if (-f $request_filename) {
fastcgi_pass 127.0.0.1:9000;
}
}
location ~ mercury {
fastcgi_param TOPINCS_STORE mercury;
}
location ~ /.[0-9].[0-9]+.0-9? {

            add_header Expires "Fri, 31 Dec 2020 23:59:59 GMT";
            add_header Cache-Control "public";
            }
    location /mercury {
             rewrite

^/mercury/([3-9].[0-9].[0-9]./(.core-topics|css|images|js|vendor|fonts).)$
/mercury/$1;
rewrite ^/mercury((.|/).*)$
/mercury/.topincs?request=$1;
}
}

/etc/php5/fpm/php.ini

[PHP]
short_open_tag = Off
asp_tags = Off
precision = 14
output_buffering = 4096
zlib.output_compression = Off
implicit_flush = Off
unserialize_callback_func =
serialize_precision = 17
disable_functions =
pcntl_alarm,pcntl_fork,pcntl_waitpid,pcntl_wait,pcntl_wifexited,pcntl_wifstopped,pcntl_wifsignaled,pcntl_wexitstatus,pcntl_wtermsig,pcntl_wstopsig,pcntl_signal,pcntl_signal_dispatch,pcntl_get_last_error,pcntl_strerror,pcntl_sigprocmask,pcntl_sigwaitinfo,pcntl_sigtimedwait,pcntl_exec,pcntl_getpriority,pcntl_setpriority,
disable_classes =
zend.enable_gc = On
expose_php = On
max_execution_time = 30
max_input_time = 60
memory_limit = 128M
error_reporting = E_ALL & ~E_DEPRECATED & ~E_STRICT
display_errors = Off
display_startup_errors = Off
log_errors = On
log_errors_max_len = 1024
ignore_repeated_errors = Off
ignore_repeated_source = Off
report_memleaks = On
track_errors = Off
html_errors = On
error_log = /var/log/php_errors.log
variables_order = “GPCS”
request_order = “GP”
register_argc_argv = Off
auto_globals_jit = On
post_max_size = 8M
auto_prepend_file =
auto_append_file =
default_mimetype = “text/html”
doc_root =
user_dir =
enable_dl = Off
cgi.fix_pathinfo=0
file_uploads = On
upload_max_filesize = 2M
max_file_uploads = 20
allow_url_fopen = On
allow_url_include = Off
default_socket_timeout = 60
request_terminate_timeout = 30s
[CLI Server]
cli_server.color = On

[Pdo_mysql]
pdo_mysql.cache_size = 2000
pdo_mysql.default_socket=

[mail function]
SMTP = localhost
smtp_port = 25
mail.add_x_header = On
[SQL]
sql.safe_mode = Off
[ODBC]
odbc.allow_persistent = On
odbc.check_persistent = On
odbc.max_persistent = -1
odbc.max_links = -1
odbc.defaultlrl = 4096
odbc.defaultbinmode = 1
[Interbase]
ibase.allow_persistent = 1
ibase.max_persistent = -1
ibase.max_links = -1
ibase.timestampformat = “%Y-%m-%d %H:%M:%S”
ibase.dateformat = “%Y-%m-%d”
ibase.timeformat = “%H:%M:%S”
[MySQL]
mysql.allow_local_infile = On
mysql.allow_persistent = On
mysql.cache_size = 2000
mysql.max_persistent = -1
mysql.max_links = -1
mysql.default_port =
mysql.default_socket =
mysql.default_host =
mysql.default_user =
mysql.default_password =
mysql.connect_timeout = 60
mysql.trace_mode = Off
[MySQLi]
mysqli.max_persistent = -1
mysqli.allow_persistent = On
mysqli.max_links = -1
mysqli.cache_size = 2000
mysqli.default_port = 3306
mysqli.default_socket =
mysqli.default_host =
mysqli.default_user =
mysqli.default_pw =
mysqli.reconnect = Off
[mysqlnd]
mysqlnd.collect_statistics = On
mysqlnd.collect_memory_statistics = Off
[bcmath]
bcmath.scale = 0
[Session]
session.save_handler = files
session.use_cookies = 1
session.use_only_cookies = 1
session.name = PHPSESSID
session.auto_start = 0
session.cookie_lifetime = 0
session.cookie_path = /
session.cookie_domain =
session.cookie_httponly =
session.serialize_handler = php
session.gc_probability = 0
session.gc_divisor = 1000
session.gc_maxlifetime = 1440
session.bug_compat_42 = Off
session.bug_compat_warn = Off
session.referer_check =
session.cache_limiter = nocache
session.cache_expire = 180
session.use_trans_sid = 0
session.hash_function = 0
session.hash_bits_per_character = 5
url_rewriter.tags =
“a=href,area=href,frame=src,input=src,form=fakeentry”
[MSSQL]
mssql.allow_persistent = On
mssql.max_persistent = -1
mssql.max_links = -1
mssql.min_error_severity = 10
mssql.min_message_severity = 10
mssql.compatability_mode = Off
mssql.secure_connection = Off
[Tidy]
tidy.clean_output = Off
[soap]
soap.wsdl_cache_enabled=1
soap.wsdl_cache_dir=“/tmp”
soap.wsdl_cache_ttl=86400
soap.wsdl_cache_limit = 5
[ldap]
ldap.max_links = -1
[dba]
cgi.fix_pathinfo = 0;

/etc/php5/fpm/pool.d/www.conf

[www]
user = www-data
group = www-data
listen = 127.0.0.1:9000
pm = dynamic
pm.max_children = 5
pm.start_servers = 2
pm.min_spare_servers = 1
pm.max_spare_servers = 3

slowlog = /var/log/php5-fpm.slow.log
request_terminate_timeout = 30s
chdir = /
php_value[include_path] =
“/home/apollo/topincs/php:/home/apollo/topincs/vendor/php”
php_value[default_charset] = “UTF-8”
php_value[magic_quotes_gpc] = “0”
php_value[php_value max_execution_time] = “7200”
php_value[php_value memory_limit] = “500M”
php_value[php_value short_open_tag] = “0”
env[TOPINCS_STORE] = “mercury”;

That’s all I can think of. If you have any ideas how to make this work
it would be GREATLY appreciated, not only by my but also by other
Topincs users, who will enjoy using nginx instead of apache.

Many thanks in advance

Piotr

P.S.

Topincs installation instructions are here:

http://www.cerny-online.com/topincs/manual/installing

–
http://okle.pl

Piotr_Kopszak · July 11, 2013, 12:51am

On Fri, Jul 05, 2013 at 03:13:51PM +0200, Piotr Kopszak wrote:

Hi there,

I have not tested any of this, so take it as “may be interesting to
consider” rather than “this is the recipe to use”.

"My main objective in the Apache and Topincs integration was that it
is possible to occupy any URL space below a domain without the need
for a general ‘root’, e.g. everything under /topincs. Plus it should
be possible to run different stores under different Topincs versions,
just in case.

I confess I don’t know what those words mean, in terms of anything that
nginx should do.

I think it says “there can be no configured prefix”; but it might mean
“the prefix can be set by the administrator to whatever they want”.

nginx works in terms of locations, which are “parts of the url
hierarchy”. It is generally good if these are segregated by url prefix.

So Apache does two things for Topincs:

The rewrite rules basically cut off the store path prefix (e.g.
/trial/movies) and pass the result on to
TOPINCS_HOME/docroot/.topincs.

Based on the path prefix it maps the URL to a store (database).
This is done by setting an Apache environment variable in the
configuration. This variable is read in .topincs only.

So once you manage to provide .topincs with the above, you are set." (

I suspect that these words make perfect sense to someone who has studied
the topincs jargon. That’s not me. But I guess that if “an Apache
environment variable” means specifically that, then it may not work
outside of apache, without some code changes.

For nginx the questions are, approximately, “what url are you
requesting”,
and “what do you want nginx to do with that request”.

If you can answer those questions, you’ll probably have a better chance
of getting a working configuration.

/etc/apache2/sites-enabled/000-default contains:

apache is very different from nginx. It allows configuration based on
the
url (location) and also the filesystem (directory, file). It combines
configuration from different blocks to determine what happens with one
request. It handles php internally.

nginx allows configuration based on the url (location). One request is
handled in one location. Unless the config you want is active in that
one location, is does not apply. php is entirely external, typically
done by talking to a fastcgi server.

So…

<VirtualHost *:80>

This is all fairly general, and pretty much doesn’t apply for nginx
(unless the defaults are insufficient).

<Directory “/home/apollo/topincs/docroot”>

If you can find the matching url, you can put this in a location{}.

DirectoryIndex index.php
AddType 'text/html; charset=UTF-8' .html
DefaultType application/x-httpd-php

Does that mean “urls ending in .html are served from the filesystem;
everything else is handled by php”? Or is it “urls ending in a specific
list of things are served from the filesystem; everything else is
handled
by php”? Or something else?

That might make a difference for the nginx config.

php_value include_path
“/home/apollo/topincs/php:/home/apollo/topincs/vendor/php”
php_value default_charset “UTF-8”

Those all look like php configuration settings. Put them in your php
config file, not your nginx one.

<Directory ~
“/home/apollo/topincs/docroot/[0-9].[0-9]+.0-9?”>

This looks like another location.

You might want to nest this inside the previous one; or you might want
to repeat what is in the previous one, here.

What configuration does topincs expect will apply for matching urls?

<LocationMatch ^/mercury/.*>
SetEnv TOPINCS_STORE mercury

What uses that SetEnv result? If it is the php application, maybe it
should be a fastcgi_param. If it is something else, then the details
might matter.

RewriteRule
^/mercury/([3-9].[0-9].[0-9]./(.core-topics|css|images|js|vendor|fonts).)$
/mercury/$1 [PT,E=TOPINCS_STORE:mercury]
RewriteRule ^/mercury((.|/).*)$ /mercury/.topincs?request=$1
[PT,L,QSA,E=TOPINCS_STORE:mercury]
Alias /mercury “/home/apollo/topincs/docroot”

And what do those lines do?

“/mercury” is your prefix. The second one seems to say that
/mercury.anything and /mercury/anything are handled by the php script
at /home/apollo/topincs/docroot/.topincs with a query string of
request=.anything or request=/anything. (Or maybe it also appends the
original query string.)

I’m not sure about the first one. /mercury/version-number/some-prefixes
are… served from the file system? Or sent through to the second one
where they match /mercury/anything?

I’m mostly guessing here, because I don’t speak (much) apache.

user www-data;
worker_processes 4;
pid /var/run/nginx.pid;
events {
worker_connections 768;
}
http {

For initial testing, keep it simple. Only add directives where you know
why they are there. And run in debug mode.

    include /etc/nginx/mime.types;

That line is useful, if you are serving something from the filesystem.

    error_log /var/log/nginx/error.log debug;

And that one, because of the debug mode.

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;

I suggest you include just the one named file that you are testing with,
to avoid any confusion. Your test system is only intended to run a
single
service, so keep it that way.

    include fastcgi_params;

That might be useful; but it is probably more useful within the server{}
or location{} that you care about.

}

server {
listen 80;
root /home/apollo/topincs/docroot;
index index.php;

Here it starts getting messy.

Rather than you trying to translate from apache to nginx, could you try
to translate from apache to a word-description of how some specific urls
should be handled? After that, it may be obvious what the nginx config
should be.

So, when you browse your apache-based topincs server, choose a few
urls that you access. You’ll see them in your browser location bar,
or in your server access log file.

When you “curl -i” those urls, what do you see? An indication that it
was served from the filesystem, or from php, or something else?

When you know that, you’ll be able to define the location{} blocks that
you will need in nginx.

And after that, you’ll have a better chance of putting in the rest of
the configuration.

Good luck with it,

f

Francis D. [email protected]

Migrating Topincs to nginx

^/mercury/([3-9].[0-9].[0-9]./(.core-topics|css|images|js|vendor|fonts).)$ /mercury/$1; rewrite ^/mercury((.|/).*)$ /mercury/.topincs?request=$1; } }

f

^/mercury/([3-9].[0-9].[0-9]./(.core-topics|css|images|js|vendor|fonts).)$
/mercury/$1;
rewrite ^/mercury((.|/).*)$
/mercury/.topincs?request=$1;
}
}