Nginx serving R scripts via CGI/FastRWeb

iamstef · February 19, 2013, 1:49pm

Hi,

Anyone here testing, experimenting with R and nginx ?
Im trying to setup nginx to serve R scripts via
CGI using Rserve, FastRWeb modules as described here:

My nginx is configured like:

    location ~ ^/cgi-bin/.*\.cgi$ {
        gzip off;
        fastcgi_pass

unix:/opt/sdr/report/ws/fastcgi_temp/nginx-fcgi.sock;
fastcgi_read_timeout 5m;
fastcgi_index index.cgi;
#fastcgi_buffers 8 4k;
#
# You may copy and paste the lines under or use include
directive
# include /etc/nginx/nginx-fcgi.conf;
# In this example all is in one file
#
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param GATEWAY_INTERFACE CGI/1.1;
fastcgi_param SERVER_SOFTWARE nginx;
fastcgi_param SCRIPT_NAME $fastcgi_script_name;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT /opt/sdr/report/docroot;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $server_name;
}

and Im using fcgiwrap from http://nginx.localdomain.pl/wiki/FcgiWrap.
CGI scripts work fine but Im not able to any R scripts probable due
R being not correctly called via nginx.conf …

I am trying something like: http://localhost/cgi-bin/R/foo.png?n=500
and R is a binary file under cgi-bin directory which should call
FastRWeb …

Any pointers, help ?

thanks,
Stefan

iamstef · February 19, 2013, 2:50pm

Does the R cgi script filename end in .cgi ? That’s what you specify, it
appears.

On Tue, Feb 19, 2013 at 7:47 AM, Stefan P.
<[email protected]

iamstef · February 19, 2013, 3:51pm

On 08:50 Tue 19 Feb , Harold Sinclair wrote:

Does the R cgi script filename end in .cgi ? That’s what you specify, it
appears.

No, it does not end with cgi since the R scripts will not be called
directly
by nginx.

As I understood I should call the R scripts like:
http://localhost/cgi-bin/R/foo where
there is a R binary file under cgi-bin which will proxy those to
FastRWeb
and Rserve R modules. So under cgi-bin there are no R scripts
whatsoever,
but they usually will go under: /opt/sdr/report/var/FastRWeb/web.R
directory.

Now my problem seems to be related how the R binary under cgi-bin will
ever be called via CGI and further FastRWeb … since my nginx.conf
knows nada
about R being called. Currently my nginx.conf is configured to run
anything ending .cgi as cgi scripts …

/opt/sdr/report/docroot/cgi-bin
$ ls -lrt
total 60
-rwxr-x— 1 sdr sdr 3715 Feb 19 12:00 initial.cgi
-rwxr-xr-x 1 sdr sdr 55932 Feb 19 13:53 R

$ file R
R: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically
linked (uses shared libs), for GNU/Linux 2.6.24,
BuildID[sha1]=0x0351c94aaf487ee3559a722446865b7ae0f3b7cc, not stripped

Here the R scripts:

/opt/sdr/report/var/FastRWeb
$ ls -lrt
total 16
drwxr-xr-x 2 sdr sdr 4096 Feb 18 00:08 web
drwxr-xr-x 2 sdr sdr 4096 Feb 18 00:08 tmp
drwxr-xr-x 2 sdr sdr 4096 Feb 19 14:27 code
srw-rw-rw- 1 sdr sdr 0 Feb 19 14:27 socket
=>> drwxr-xr-x 2 sdr sdr 4096 Feb 19 14:28 web.R

all R scripts will go here web.R directory.

stefan

iamstef · February 19, 2013, 5:41pm

On 11:22 Tue 19 Feb , Harold Sinclair wrote:

Sounds like you ought to try writing a wrapper script ending in .cgi in the
cgi dir that grabs the query string and hands the job off to the R
executable. Not sure if R is required in the cgi-bin directory. You might
have to enable symlinking out for it to work.

yeah not sure how to do it. FastRWeb tell us:

" FastRWeb consists of several parts:

Webserver-to-R pipeline, consisting of either a thin CGI or a PHP

client connecting to Rserve which sources and runs the R script. The CGI
client is called Rcgi and is compiled as part of the package. The PHP
client is part of Rserve in the clients section.
"

So Im on option 1: CGI. The client will be this binary which somehow
needs to be
executed for each R script. I placed the Rcgi under cgi.bin directory
and rename it
as R.cgi.

If I call directly:

http://localhost:9001/cgi-bin/R.cgi the browser returns:
Error: no function or path specified.

http://localhost:9001/cgi-bin/R.cgi/foo.png?n=100
I see under error log:
2013/02/19 18:39:53 [error] 2847#0: *56 open()
“/opt/sdr/report/docroot/cgi-bin/R.cgi/foo.png” failed (20: Not a
directory), client: 127.0.0.1, server: sdrrep, request: “GET
/cgi-bin/R.cgi/foo.png?n=100 HTTP/1.1”, host: “localhost:9001”

stefan

iamstef · February 19, 2013, 5:22pm

Sounds like you ought to try writing a wrapper script ending in .cgi in
the
cgi dir that grabs the query string and hands the job off to the R
executable. Not sure if R is required in the cgi-bin directory. You
might
have to enable symlinking out for it to work.

On Tue, Feb 19, 2013 at 9:49 AM, Stefan P.
<[email protected]

iamstef · February 19, 2013, 5:44pm

So Im on option 1: CGI. The client will be this binary which somehow needs to be
executed for each R script. I placed the Rcgi under cgi.bin directory and rename
it
as R.cgi.

and I forgot to mention for CGI Im using FcgiWrap:
http://nginx.localdomain.pl/wiki/FcgiWrap

stefan

iamstef · February 19, 2013, 6:24pm

On Tue, Feb 19, 2013 at 12:47:23PM +0000, Stefan P. wrote:

Hi there,

This location:

    location ~ ^/cgi-bin/.*\.cgi$ {

will only match some requests that end in “.cgi” (before the ?, if
that applies).

What you possibly want is a separate location just for your “R”
requests.

Something like (untested by me!)

location ^~ /cgi-bin/R/ { }

in which you have your “fastcgi_pass” directive, plus whatever
“fastcgi_param” directives you need to make it work. That is probably

fastcgi_param REQUEST_URI $request_uri;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param SCRIPT_FILENAME $document_root/cgi-bin/R;

but the exact details depend on what your fastcgi server wants.

(You may want to set PATH_INFO, if your application uses that.)

See Module ngx_http_core_module for the details of which one location{}
is chosen for each request, and then see the fastcgi documentation to
decide exactly what params you need.

Usually, SCRIPT_FILENAME is “the file on the filesystem that the fastcgi
server should execute”; and other things are “stuff that the fastcgi
server or the application can use to decide what to do”.

and Im using fcgiwrap from http://nginx.localdomain.pl/wiki/FcgiWrap.
CGI scripts work fine but Im not able to any R scripts probable due
R being not correctly called via nginx.conf …

In this case, yes. Your nginx configuration was such that your requests
for “R” were not being sent to the fastcgi server.

I am trying something like: http://localhost/cgi-bin/R/foo.png?n=500
and R is a binary file under cgi-bin directory which should call
FastRWeb …

Provided that the fastcgi server is able to run whatever file you name
in
SCRIPT_FILENAME, and has access to whatever other params it cares about,
something like the above configuration has a chance of working.

f

Francis D. [email protected]

iamstef · February 20, 2013, 2:25pm

On Wed, Feb 20, 2013 at 10:00:04AM +0000, Stefan P. wrote:

Something like (untested by me!)
location ^~ /cgi-bin/R/ { }

yep correct. I do have now on cgi-bin directory a binary file called: R.
I did change my nginx.conf to have a new location definition for the
R calls, like here:
    location ~ ^/cgi-bin/R$ {

“~” means “regex”.

“^” means “start of string”.

“$” means “end of string”.

This location will only match requests that are

/cgi-bin/R

or

/cgi-bin/R?something

and not

/cgi-bin/R/something

But when I try: http://localhost:9001/cgi-bin/R/foo?n=50
this returns with same error that there is no such directory, seems somehow
the request it is not passed via CGI to FastWebR.

Yes, that’s what you configured.

What happens when you test exactly what was suggested?

f

Francis D. [email protected]

iamstef · February 20, 2013, 11:02am

Something like (untested by me!)
location ^~ /cgi-bin/R/ { }

yep correct. I do have now on cgi-bin directory a binary file called: R.
I did change my nginx.conf to have a new location definition for the
R calls, like here:

    location ~ ^/cgi-bin/R$ {
        gzip off;
        fastcgi_pass

unix:/opt/sdr/report/ws/fastcgi_temp/nginx-fcgi.sock;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param SCRIPT_FILENAME
/opt/sdr/report/docroot$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;

        fastcgi_param    REQUEST_URI  $request_uri;
        fastcgi_param    DOCUMENT_URI  $document_uri;
        fastcgi_param    DOCUMENT_ROOT  /opt/sdr/report/docroot;
        fastcgi_param    SERVER_PROTOCOL  $server_protocol;
        fastcgi_param    REMOTE_ADDR  $remote_addr;
        fastcgi_param    REMOTE_PORT  $remote_port;
        fastcgi_param    SERVER_ADDR  $server_addr;
        fastcgi_param    SERVER_PORT  $server_port;
        fastcgi_param    SERVER_NAME  $server_name;
    }

This works if I put in my browser:
http://localhost:9001/cgi-bin/R I get some error in the browser but it
does
seem to work since I did not input any parameters to R call.

But when I try: http://localhost:9001/cgi-bin/R/foo?n=50
this returns with same error that there is no such directory, seems
somehow
the request it is not passed via CGI to FastWebR.

2013/02/20 11:52:01 [error] 3637#0: *9 open()
“/opt/sdr/report/docroot/cgi-bin/R/foo” failed (20: Not a directory),
client:
127.0.0.1, server: sdrrep, request: “GET /cgi-bin/R/foo?n=50 HTTP/1.1”,
host:
“localhost:9001”

In this case, yes. Your nginx configuration was such that your requests
for “R” were not being sent to the fastcgi server.

And still they are not passed correctly.
something still messed up with my nginx.conf

you can see entire nginx.conf here:
http://www.systemdatarecorder.org/nginx.conf

Stefan

iamstef · February 20, 2013, 9:30pm

/cgi-bin/R/something

yep. silly me, I do have a location for all /cgi-bin/R/ calls, like:

    location ~ ^/cgi-bin/R/ {
        gzip off;
        fastcgi_pass

unix:/opt/sdr/report/ws/fastcgi_temp/nginx-fcgi.sock;
fastcgi_index index.cgi;
fastcgi_param SCRIPT_FILENAME
/opt/sdr/report/docroot/$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_param DOCUMENT_URI $document_uri;
fastcgi_param DOCUMENT_ROOT /opt/sdr/report/docroot;
}

and another one for anything *.cgi:

    location ~ ^/cgi-bin/.*\.cgi$ {
        gzip off;
        fastcgi_pass

unix:/opt/sdr/report/ws/fastcgi_temp/nginx-fcgi.sock;
fastcgi_read_timeout 5m;
fastcgi_index index.cgi;
#fastcgi_buffers 8 4k;
#
# You may copy and paste the lines under or use include
directive
# include /etc/nginx/nginx-fcgi.conf;
# In this example all is in one file
…
}

since under my /cgi-bin/ I can have lots of cgi scripts and the
executable R cgi script.

Testing any calls for /cgi-bin/R/foo returns:
Cannot get script name, are DOCUMENT_ROOT and SCRIPT_NAME (or
SCRIPT_FILENAME) set and is the script executable?
Cannot get script name, are DOCUMENT_ROOT and SCRIPT_NAME (or
SCRIPT_FILENAME) set and is the script executable?

Im trying to see if fastcgi_split_path_info might help anything.

stefan

iamstef · February 20, 2013, 10:44pm

On Wed, Feb 20, 2013 at 08:28:27PM +0000, Stefan P. wrote:

    location ~ ^/cgi-bin/R/ {

That will match anything starting with /cgi-bin/R/, which is (most of)
what you want.

These requests will…

        fastcgi_pass  unix:/opt/sdr/report/ws/fastcgi_temp/nginx-fcgi.sock;

be sent to the fastcgi server.

Now, for all of these requests, you want to tell the fastcgi server to
use the file /opt/sdr/report/docroot/cgi-bin/R. So:

        fastcgi_index index.cgi;

you don’t need that, since it will never apply; and

        fastcgi_param SCRIPT_FILENAME

/opt/sdr/report/docroot/$fastcgi_script_name;

that should be something that expands to
/opt/sdr/report/docroot/cgi-bin/R. Which is probably
$document_root/cgi-bin/R; you can use whatever matches exactly that
filename. $fastcgi_script_name does not have the correct value by
default,
in this case.

        fastcgi_param PATH_INFO $fastcgi_path_info;

If your application uses PATH_INFO, then you’ll want to set it something
like that. I don’t think that $fastcgi_path_info actually has a value
by default, though.

        fastcgi_param QUERY_STRING     $query_string;
        fastcgi_param REQUEST_METHOD   $request_method;
        fastcgi_param CONTENT_TYPE     $content_type;
        fastcgi_param CONTENT_LENGTH   $content_length;
        fastcgi_param REQUEST_URI        $request_uri;
        fastcgi_param DOCUMENT_URI       $document_uri;
        fastcgi_param DOCUMENT_ROOT  /opt/sdr/report/docroot;

They are all fairly standard; probably only the first and last matter,
but it depends on your fastcgi server and your application. The last
one would usually be written to use $document_root, but anything that
ends up with the correct value is good.

Note that your fastcgi server, from the logs you provide, either wants
SCRIPT_FILENAME to be correct, or DOCUMENT_ROOT and SCRIPT_NAME to be
correct; and you don’t provide a SCRIPT_NAME.

That may not matter when SCRIPT_FILENAME points to the R binary.

Testing any calls for /cgi-bin/R/foo returns:
Cannot get script name, are DOCUMENT_ROOT and SCRIPT_NAME (or SCRIPT_FILENAME)
set and is the script executable?
Cannot get script name, are DOCUMENT_ROOT and SCRIPT_NAME (or SCRIPT_FILENAME)
set and is the script executable?

From the above config, SCRIPT_FILENAME was probably
/opt/sdr/report/docroot/cgi-bin/R/foo, which is not an executable file.

Im trying to see if fastcgi_split_path_info might help anything.

Yes.

http://nginx.org/r/fastcgi_split_path_info

It will change the values of $fastcgi_script_name and
$fastcgi_path_info.

f

Francis D. [email protected]