FRiCKLE pres. ngx_postgres

Hello guys,
I’m pleased to announce that ngx_postgres is finally publicly available!

Website and distfiles are available at:
http://labs.frickle.com/nginx_ngx_postgres/

GitHub repository is available at:

Best regards,
Piotr S. < [email protected] >

Good Jobs! :slight_smile:

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,82992,83190#msg-83190

Hello!
Awesome job, really.

I can imagine one scenario where similar plugins (polling backends for
info) would be useful, where nginx can act as a traffic router,
something like:
$backend = select backend from route_table where host=’$http_host’

proxy_pass http://$backend

By looking at the docs I’m guessing this isn’t possible at the moment
(as well as the docs for ngx_drizzle, …) - but are there any plans for
similar behavior?

Cheers,
Johan

Hello,

I can imagine one scenario where similar plugins (polling backends for
info) would be useful, where nginx can act as a traffic router, something
like:
$backend = select backend from route_table where host=’$http_host’

proxy_pass http://$backend

By looking at the docs I’m guessing this isn’t possible at the moment (as
well as the docs for ngx_drizzle, …) - but are there any plans for
similar behavior?

This is indeed good use case for ngx_databases. I was solving such
problem
in the past (before ngx_drizzle was released), but I totally forgot
about it
while developing ngx_postgres. Thanks for the reminder :wink:

Anyway, I’ve just pushed changes into ngx_postgres’s GitHub repository
[1]
and this is now possible. Please note that you’ll need agentzh’s fork of
eval module [2] to be able to use it with ngx_postgres.

This, along with some other changes, will be released as
ngx_postgres-0.2
early next week.

location / {
eval_subrequest_in_memory off;

eval $backend {
    postgres_pass       database;
    postgres_query      "select * from backends limit 1";
    postgres_get_value  0 0;
}

proxy_pass $backend;

}

[1] http://github.com/FRiCKLE/ngx_postgres
[2] http://github.com/agentzh/nginx-eval-module

Best regards,
Piotr S. < [email protected] >

Hey,

Ah, I love that postgres_get_value directive! :smiley:

:slight_smile:

But to be honest, I’ve been planning a ngx_rds_picker filter module
that can do such “cell picking” in the result sets on the RDS level so
that specific DB drivers don’t have to do it themselves :wink:

This seems like a more flexible way to do things, but I don’t think that
generating whole RDS response just to get single value out of it is the
best
idea :wink:

Of course, ideally “cell picking” should be done on the query level, so
that
database would return only the value that we are really interested in…
But
then comes the caching, so…

Having said that, I’m looking forward to more of your RDS filters :slight_smile:

Best regards,
Piotr S. < [email protected] >

Version 0.2 is now available at:
http://labs.frickle.com/nginx_ngx_postgres/

GitHub repository is available at:
http://github.com/FRiCKLE/ngx_postgres

Changes:
2010-05-12 VERSION 0.2
* Add various improvements to build and testing infrastructures.
Mostly done by Yichun Z. (agentzh).

* Put more restrictions on "postgres_pass" and "postgres_query"
  directives. Handle their bad configuration properly.

2010-05-10
* Log PostgreSQL errors into error.log.
Reminded by Yichun Z. (agentzh).

2010-05-06
* Remove connection timeout from re-used keepalive connection.

* Fix libpq headers detection on Debian.
  Patch from Weibin Y..

* Add "postgres_get_value" directive.
  Requested by Johan Bergstroem.

* Bring back fail-safe check that got lost during pre-release
  refactorization. Without this check performance was reduced
  few times under high load, because about 1% of keepalive
  connections got disconnected.

Best regards,
Piotr S. < [email protected] >

2010/5/7 Piotr S. [email protected]:

eval $backend {
postgres_pass database;
postgres_query “select * from backends limit 1”;
postgres_get_value 0 0;
}

Ah, I love that postgres_get_value directive! :smiley:

But to be honest, I’ve been planning a ngx_rds_picker filter module
that can do such “cell picking” in the result sets on the RDS level so
that specific DB drivers don’t have to do it themselves :wink:

Cheers,
-agentzh

Version 0.3 is now available at:
http://labs.frickle.com/nginx_ngx_postgres/

GitHub repository is available at:
http://github.com/FRiCKLE/ngx_postgres

Changes:
2010-06-13 VERSION 0.3
* Allow configuration of method-specific queries.

* Restrict "postgres_pass" directive to "location" context.

2010-06-07
* Free keepalive connections on nginx shutdown.
Requested by Yichun Z. (agentzh).

* Fix memory leak that was happening when nginx was configured
  to use non-existing database tables, etc.
  Found by Valgrind, reported by Yichun Z. (agentzh).

2010-06-04
* Use recently standardized error codes in RDS format.

2010-06-03
* Allow request methods other than GET and HEAD.
From Yichun Z. (agentzh) via ngx_drizzle.

Best regards,
Piotr S. < [email protected] >

Version 0.5 is now available at:
http://labs.frickle.com/nginx_ngx_postgres/

GitHub repository is available at:
http://github.com/FRiCKLE/ngx_postgres

Changes:
2010-08-03 VERSION 0.5
* Fix compatibility with nginx-0.8.47+.

2010-07-20
* Add “postgres_escape” directive.

Best regards,
Piotr S. < [email protected] >

Hi guys,
I’m thrilled to announce new release of ngx_postgres.

This is MAJOR update focused on new features and interoperability with
number of 3rd party modules (including: ngx_http_auth_request_module,
nginx-eval-module and ngx_lua). When combined with ngx_form_input or
ngx_lua
(for extracting values from client’s request body), this module allows
one
to create fully RESTful webservices using only nginx.conf! And it’s
production-ready :slight_smile:

This release is accompanied by the release of ngx_rds_json v0.10, which
produces JSON responses for all 2xx status codes. This release is pretty
much required for people wanting to use “postgres_rewrite” directive.

Distfiles and documentation are available at:
http://labs.frickle.com/nginx_ngx_postgres/

GitHub repository is available at:
http://github.com/FRiCKLE/ngx_postgres

Changes:
2010-07-05 VERSION 0.4
* Optimize generation of RDS output.

2010-07-02
* Fix serious bug that under certain conditions (query evaluated
to empty string, failed connection to the database, etc) would
lead to segmentation fault on versions older than nginx-0.8.17
(including nginx-0.7.x).

2010-06-30
* When returning row or value, use Content-Type specified by
“default_type” directive instead of “text/plain”.

* Allow column to be specified by its name instead of its number
  (in "postgres_output" and "postgres_set" directives).

2010-06-23
* Add “postgres_rewrite” directive.

* Add "$postgres_affected" variable.

2010-06-22
* Fix issue that would stop gzip filter from processing
responses in RDS format.
Found by Qing Lin (kindy), fixed by Yichun Z. (agentzh).

2010-06-21
* Add “postgres_output” directive.

2010-06-18
* Add “$postgres_query” variable.

2010-06-16
* Add “postgres_set” directive.

* Add "$postgres_columns" and "$postgres_rows" variables.

Best regards,
Piotr S. < [email protected] >

Version 0.6 is now available at:
http://labs.frickle.com/nginx_ngx_postgres/

GitHub repository is available at:
http://github.com/FRiCKLE/ngx_postgres

2010-08-15 VERSION 0.6
* Fix linking issue that manifested itself when nginx was build
with both: ngx_postgres and ngx_supervisord modules.
Reported by Sergey A. Osokin.

2010-08-09
* Fix pointer signedness mismatch, which broke build on Darwin
and probably few other operating systems.
Reported by sahuguet, fixed by Yichun Z. (agentzh).

Best regards,
Piotr S. < [email protected] >

Version 0.7 is now available at:
http://labs.frickle.com/nginx_ngx_postgres/

GitHub repository is available at:
http://github.com/FRiCKLE/ngx_postgres

Changes:
2010-09-30 VERSION 0.7
* Add option to send original response body with error responses
set by “postgres_rewrite” directive.

2010-08-25
* Fix error that could lead to failed connection to the database.

* Log more details on failed connection to the database.

Best regards,
Piotr S. < [email protected] >

On Thu, Dec 23, 2010 at 11:27 AM, Piotr S.
[email protected] wrote:

2010-12-23 VERSION 0.8

  • Add option to return content in binary format using
    “binary_value” output format in “postgres_output” directive.
    Mostly done by Yichun Z. (agentzh).

Just a brief note on this:

I originally introduced this just because Qunar.com (the biggest
air-ticket search engine here in China) is storing (small) images into
their PostgreSQL databases, and they’d like to serve out the images
directly to the web browsers, and without introducing any extra middle
layers to maximize speed.

2010-11-01

  • Support “postgres_pass”, “postgres_query”, “postgres_rewrite”
    and “postgres_output” directives in “if” pseudo-locations.
    From Yichun Z. (agentzh).

This allows simple code branches by using nginx’s if mechanism. And
yeah, nginx’s “if” is evil but it can make simple things easy. It’s
recommended to use ngx_lua’s set_by_lua and other stuffs to do
complicated code branching though :slight_smile:

Cheers,
-agentzh

Version 0.8 is now available at:
http://labs.frickle.com/nginx_ngx_postgres/

GitHub repository is available at:
http://github.com/FRiCKLE/ngx_postgres

Changes:
2010-12-23 VERSION 0.8
* Add option to return content in binary format using
“binary_value” output format in “postgres_output” directive.
Mostly done by Yichun Z. (agentzh).

2010-11-01
* Support “postgres_pass”, “postgres_query”, “postgres_rewrite”
and “postgres_output” directives in “if” pseudo-locations.
From Yichun Z. (agentzh).

2010-10-02
* Major rewrite of “postgres_escape” directive.

Best regards,
Piotr S. < [email protected] >