# Reading from a named pipe / fifo

**URL:** <https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972>\
**Category:** NGINX\
**Created:** [January 20, 2012, 10:37pm UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972 "2012-01-20T22:37:11Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![dubstep](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cdc98d/32.png) [@dubstep](https://www.ruby-forum.com/u/dubstep)\
**Post date:** [January 20, 2012, 10:37pm UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972/1 "2012-01-20T22:37:11Z")

</div>

Hi,

how could I configure (or change) nginx to serve the output  
from a named pipe / fifo (that was created with mknod or mkfifo)  
as if it were a regular HTML file?

Let’s say I have a fifo named fifo.html that gets input from an  
application. This input will sometimes be read locally by another  
application and sometimes remotely through nginx, but once the  
information is read, it should be considered consumed (no longer  
available).

Can the current version of nginx be configured to provide this  
feature?

Limiting the number of times a certain file can be served would  
also be an acceptable solution.

If this isn’t possible, could you point me in the right direction  
so I could change the source code or write a module that would  
allow nginx to serve output from named pipes / fifos?

Thank you,  
Max

---

<div class="post-metadata">

**Author:** ![max](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@max](https://www.ruby-forum.com/u/max)\
**Post date:** [January 20, 2012, 11:21pm UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972/2 "2012-01-20T23:21:45Z")

</div>

nginx can http proxy to a unix socket. (as well as fastcgi, uwsgi, etc).  
So  
could the other app provide the data that way? With a fifo, you’d need  
a  
way to specify when to stop reading, so you may as well use a known  
protocol.

---

<div class="post-metadata">

**Author:** ![max](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@max](https://www.ruby-forum.com/u/max)\
**Post date:** [January 22, 2012, 10:30am UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972/3 "2012-01-22T10:30:41Z")

</div>

21 января 2012, 02:21 от Brian A. : \> nginx can http proxy to a unix  
socket. (as well as fastcgi, uwsgi, etc). So \> could the other app  
provide the data that way? With a fifo, you’d need a \> way to specify  
when to stop reading, so you may as well use a known \> protocol.  
Unfortunately, the other app can provide the data only through a fifo  
and there’s no easy way to change it due to licensing, so writing a fifo  
module for nginx would be the easier and cleaner solution. Igor and  
other developers, could you please give me a few tips to help me get  
started with the development of the fifo module? Thank you, Max

---

<div class="post-metadata">

**Author:** ![max](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@max](https://www.ruby-forum.com/u/max)\
**Post date:** [January 22, 2012, 11:44am UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972/4 "2012-01-22T11:44:13Z")

</div>

## Max Wrote:

> Unfortunately,  
> the other app can provide the data only through a  
> fifo and there’s no easy way to change it due to  
> licensing, so writing a fifo module for nginx  
> would be the easier and cleaner solution. Igor and  
> other developers, could you please give me a few  
> tips to help me get started with the development  
> of the fifo module? Thank you,  
> Max

Personally, the way i’d do it is to write a bit of php (or python, perl,  
etc) called via fast cgi to read your fifo (be mindful of the blocking  
nature of pipes which can cause such things to hang waiting for input).

but,

> **[Squirrels Hate Robots - PegasusGate](https://pegasusgate.net/+bob)**
>
> Bob's Tech Corner

is an example of reading a fifo from php. And it would be quite easy to  
do something like this (kinda off the top of my head):

```
    location /location/of/my/fifo.html {
            include /etc/nginx/fastcgi_params;
            fastcgi_pass

```

unix:/var/run/php-fastcgi/php-fastcgi.socket;  
fastcgi\_param SCRIPT\_FILENAME  
/path/to/local/php/that/reads/pipe.php;  
}

within the nginx config…

Posted at Nginx Forum:  
[http://forum.nginx.org/read.php?2,221499,221507#msg-221507](http://forum.nginx.org/read.php?2,221499,221507#msg-221507)

---

<div class="post-metadata">

**Author:** ![max](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@max](https://www.ruby-forum.com/u/max)\
**Post date:** [January 22, 2012, 7:58pm UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972/5 "2012-01-22T19:58:33Z")

</div>

21 января 2012, 21:41 от “takigama” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> Personally, the way i’d do it is to write a bit of php (or python, perl,  
> etc) called via fast cgi […]

Thanks for the suggestion, but using FastCGI is not an option for  
many reasons. The most elegant solution I could come up with  
without hacking nginx involves embedded Perl like this:

===\<FIFOReader.pm\>===  
use nginx;

sub handler {  
my $r = shift;  
alarm 5;  
open(FIFO, “\</path/to/fifo”) or return 444;  
while () {  
$r-\>print($\_);  
}  
$r-\>rflush();  
close(FIFO);  
return OK;  
}  
===\</FIFOReader.pm\>===

===\<nginx.conf\>===  
http {  
perl\_modules perl.modules/;  
perl\_require FIFOReader.pm;

server {  
location ~ ^/readfifo {  
ssi on;  
perl FIFOReader::handler;  
}  
}  
}  
===\</nginx.conf\>===

======

======

FIFOReader.pm is located in $nginx\_conf\_dir/perl.modules/  
readfifo is located in $document\_root/

In the FIFOReader Perl module I first set the alarm to allow  
the blocking open() call 5 seconds to open the FIFO. After  
5 seconds the blocking open() call gets interrupted and  
nginx returns code 444, which tears down the connection  
without sending any kind of response code, so there is  
no need to check for response codes - the client gets  
either nothing or the data from the FIFO.

If the blocking open() call succeeds before the timeout,  
the alarm is turned off and the data is read from  
the FIFO and passed on to the client, after which the  
FIFO is closed and the OK code is returned by nginx.

There is no additional alarm to prevent read timeouts  
(before while ()) because that is left up to the  
client by design.

Another side benefit of this approach is that it doesn’t  
require additional processes (such as FastCGI wrappers),  
so there is no need to supervise additional processes to  
make sure they’re around. The embedded Perl module  
will work for as long as nginx is running.

I have started working on a FIFO module to allow per-location  
FIFO opening and reading. If I get it to production quality I’ll  
post my patches to the list.

Max

---

<div class="post-metadata">

**Author:** ![max](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@max](https://www.ruby-forum.com/u/max)\
**Post date:** [January 22, 2012, 8:01pm UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972/6 "2012-01-22T20:01:39Z")

</div>

21 января 2012, 21:41 от “takigama” [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> Personally, the way i’d do it is to write a bit of php (or python, perl,  
> etc) called via fast cgi […]

Thanks for the suggestion, but using FastCGI is not an option for  
many reasons. The most elegant solution I could come up with  
without hacking nginx involves embedded Perl like this:

===\<FIFOReader.pm\>===  
package FIFOReader;

use nginx;

sub handler {  
my $r = shift;  
alarm 5;  
open(FIFO, “\</path/to/fifo”) or return 444;  
while () {  
$r-\>print($\_);  
}  
$r-\>rflush();  
close(FIFO);  
return OK;  
}

1;  
**END**  
===\</FIFOReader.pm\>===

===\<nginx.conf\>===  
http {  
perl\_modules perl.modules/;  
perl\_require FIFOReader.pm;

server {  
location ~ ^/readfifo {  
ssi on;  
perl FIFOReader::handler;  
}  
}  
}  
===\</nginx.conf\>===

======

======

FIFOReader.pm is located in $nginx\_conf\_dir/perl.modules/  
readfifo is located in $document\_root/

In the FIFOReader Perl module I first set the alarm to allow  
the blocking open() call 5 seconds to open the FIFO. After  
5 seconds the blocking open() call gets interrupted and  
nginx returns code 444, which tears down the connection  
without sending any kind of response code, so there is  
no need to check for response codes - the client gets  
either nothing or the data from the FIFO.

If the blocking open() call succeeds before the timeout,  
the alarm is turned off and the data is read from  
the FIFO and passed on to the client, after which the  
FIFO is closed and the OK code is returned by nginx.

There is no additional alarm to prevent read timeouts  
(before while ()) because that is left up to the  
client by design.

Another side benefit of this approach is that it doesn’t  
require additional processes (such as FastCGI wrappers),  
so there is no need to supervise additional processes to  
make sure they’re around. The embedded Perl module  
will work for as long as nginx is running.

I have started working on a FIFO module to allow per-location  
FIFO opening and reading. If I get it to production quality I’ll  
post my patches to the list.

Max

---

<div class="post-metadata">

**Author:** ![max](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@max](https://www.ruby-forum.com/u/max)\
**Post date:** [January 22, 2012, 8:29pm UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972/7 "2012-01-22T20:29:33Z")

</div>

On Jan 21, 2012, at 9:04 PM, Max wrote:

> Thanks for the suggestion, but using FastCGI is not an option for  
> many reasons. The most elegant solution I could come up with  
> without hacking nginx involves embedded Perl like this:

Won’t embedded perl block in this case? I’m not as familiar with it. If  
it does, this seems like a sub-optimal solution. FastCGI would not block  
on the nginx side. It’s pretty trivial to manage fastcgi processes.

It doesn’t have to be fastcgi, it could be a min-http server uwsgi or  
whatever or is there some restriction that nginx must talk directly to  
the fifo?

---

<div class="post-metadata">

**Author:** ![max](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@max](https://www.ruby-forum.com/u/max)\
**Post date:** [January 22, 2012, 8:07pm UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972/8 "2012-01-22T20:07:52Z")

</div>

21 января 2012, 21:41 от “takigama” :

> Personally, the way i’d do it is to write a bit of php (or python, perl,  
> etc) called via fast cgi […]

Thanks for the suggestion, but using FastCGI is not an option for  
many reasons. The most elegant solution I could come up with  
without hacking nginx involves embedded Perl like this:

===\<FIFOReader.pm\>===  
package FIFOReader;

use nginx;

sub handler {  
my $r = shift;  
alarm 5;  
open(FIFO, “\</path/to/fifo”) or return 444;  
alarm 0;  
while () {  
$r-\>print($\_);  
}  
$r-\>rflush();  
close(FIFO);  
return OK;  
}

1;  
**END**  
===\</FIFOReader.pm\>===

===\<nginx.conf\>===  
http {  
perl\_modules perl.modules/;  
perl\_require FIFOReader.pm;

server {  
location ~ ^/readfifo {  
ssi on;  
perl FIFOReader::handler;  
}  
}  
}  
===\</nginx.conf\>===

======

======

FIFOReader.pm is located in $nginx\_conf\_dir/perl.modules/  
readfifo is located in $document\_root/

In the FIFOReader Perl module I first set the alarm to allow  
the blocking open() call 5 seconds to open the FIFO. After  
5 seconds the blocking open() call gets interrupted and  
nginx returns code 444, which tears down the connection  
without sending any kind of response code, so there is  
no need to check for response codes - the client gets  
either nothing or the data from the FIFO.

If the blocking open() call succeeds before the timeout,  
the alarm is turned off and the data is read from  
the FIFO and passed on to the client, after which the  
FIFO is closed and the OK code is returned by nginx.

There is no additional alarm to prevent read timeouts  
(before while ()) because that is left up to the  
client by design.

Another side benefit of this approach is that it doesn’t  
require additional processes (such as FastCGI wrappers),  
so there is no need to supervise additional processes to  
make sure they’re around. The embedded Perl module  
will work for as long as nginx is running.

I have started working on a FIFO module to allow per-location  
FIFO opening and reading. If I get it to production quality I’ll  
post my patches to the list.

Max

---

<div class="post-metadata">

**Author:** ![max](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@max](https://www.ruby-forum.com/u/max)\
**Post date:** [January 22, 2012, 9:51pm UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972/9 "2012-01-22T21:51:18Z")

</div>

On Jan 21, 2012, at 10:35 PM, Max wrote:

> It will block, but not indefinitely, only for as long as I want it to block.  
> That’s why I used alarm to give open() 5 seconds to succeed

5 seconds can be an eternity in internet time.

> Exactly, there are strict MAC (Mandatory Access Control) restrictions  
> in place.

You can use MAC with a fastcgi script.

> Even if there weren’t, I’d still prefer the direct approach, it’s  
> much simpler and cleaner, IMHO.

I guess we’ll just have to disagree about that. Doing things to make  
nginx block for seconds at a time on purpose seems much worse than  
managing a simple fastcgi script. IMNSHO.

–Brian

---

<div class="post-metadata">

**Author:** ![max](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e5b9ba/32.png) [@max](https://www.ruby-forum.com/u/max)\
**Post date:** [January 22, 2012, 9:38pm UTC](https://www.ruby-forum.com/t/reading-from-a-named-pipe-fifo/215972/10 "2012-01-22T21:38:57Z")

</div>

22 января 2012, 06:26 от Brian A. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> Won’t embedded perl block in this case? I’m not as familiar with it. If it  
> does, this seems like a sub-optimal solution. FastCGI would not block on the  
> nginx side. It’s pretty trivial to manage fastcgi processes.

It will block, but not indefinitely, only for as long as I want it to  
block.  
That’s why I used alarm to give open() 5 seconds to succeed - in case  
it fails to open the FIFO within 5 seconds, it gets interrupted  
(through SIGALRM) and since it fails the “or return 444” part  
returns control to nginx to close the client’s connection.

However, if the open() call succeeds within 5 seconds, the alarm  
gets turned off (alarm 0) and then the FIFO is read from.

> It doesn’t have to be fastcgi, it could be a min-http server uwsgi or whatever  
> or is there some restriction that nginx must talk directly to the fifo?

Exactly, there are strict MAC (Mandatory Access Control) restrictions  
in place. Even if there weren’t, I’d still prefer the direct approach,  
it’s  
much simpler and cleaner, IMHO. Why would you want to use another  
wrapper process and yet another process to supervise the wrapper  
process and yet another process to read the FIFO and at least  
another pair of sockets when you don’t have to?

Max