# Nginx, Lua and blocking libraries

**URL:** <https://www.ruby-forum.com/t/nginx-lua-and-blocking-libraries/238425>\
**Category:** NGINX\
**Created:** [January 9, 2014, 2:36pm UTC](https://www.ruby-forum.com/t/nginx-lua-and-blocking-libraries/238425 "2014-01-09T14:36:07Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![musicdenotation](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/ea666f/32.png) [@musicdenotation](https://www.ruby-forum.com/u/musicdenotation)\
**Post date:** [January 9, 2014, 2:36pm UTC](https://www.ruby-forum.com/t/nginx-lua-and-blocking-libraries/238425/1 "2014-01-09T14:36:07Z")

</div>

Hello

I’m considering the possibility of implementing a project using Nginx  
and the Lua module. One of the requirements of the project is that the  
code must use an embedded database such as SQLite. However, as known,  
using the lua-sqlite3 library directly is not optimal because it would  
block the Nginx worker process.

My question is, is there a way to work around this in any way? For  
example, creating a coroutine to run the lua-sqlite3 calls?

If not, does anyone know of some other embedded database that works well  
with the Lua Nginx module?

Thank you in advance,  
Andre

---

<div class="post-metadata">

**Author:** ![reflex](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/f05b48/32.png) [@reflex](https://www.ruby-forum.com/u/reflex)\
**Post date:** [January 9, 2014, 6:33pm UTC](https://www.ruby-forum.com/t/nginx-lua-and-blocking-libraries/238425/2 "2014-01-09T18:33:57Z")

</div>

Hello!

On Thu, Jan 9, 2014 at 5:35 AM, Andre N. wrote:

> However, as known,  
> using the lua-sqlite3 library directly is not optimal because it would  
> block the Nginx worker process.

Well, I suggest you benchmark the actual performance and measure the  
actual blocking effect (We actually have systemtap-based tools to  
measure the epoll loop blocking effect:  
[GitHub - openresty/stapxx: Simple macro language extentions to systemtap](https://github.com/agentzh/stapxx#epoll-loop-blocking-distr) and the  
off-CPU flamegraph tool is very useful for determining the  
contributors:  
[GitHub - openresty/openresty-systemtap-toolkit: Real-time analysis and diagnostics tools for OpenResty (including NGINX, LuaJIT, ngx\_lua, and more) based on SystemTap](https://github.com/agentzh/nginx-systemtap-toolkit#sample-bt-off-cpu)  
).

Basically if your database is in-memory, then the blocking effect  
should be much smaller because no disk IO involved.

Even if you’re using on-disk database, the blocking effect should be  
quite small if your disks are fast (like modern SSD cards) and/or the  
kernel’s page cache’s hit rate is high enough.

Nginx core’s popular in-file http cache (used by proxy\_cache,  
fastcgi\_cache, and etc) also involves blocking file IO system calls  
and people are still enjoying it a lot 😉

It’s worth noting that when you’re using ngx\_lua to embed a database  
like SQLite, you should always cache the file descriptor to save  
`open` and `close` system calls.

> My question is, is there a way to work around this in any way?

Ideally you should use (or write) a pthread based network service  
frontend (be it TCP or just unix domain sockets) for SQLite and let  
your Nginx talk to this network interface in a 100% nonblocking way.

> For  
> example, creating a coroutine to run the lua-sqlite3 calls?

Basically Lua coroutines cannot work around blocking system calls. You  
need real OS threads for that.

Regards,  
-agentzh

---

<div class="post-metadata">

**Author:** ![reflex](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/f05b48/32.png) [@reflex](https://www.ruby-forum.com/u/reflex)\
**Post date:** [January 9, 2014, 7:23pm UTC](https://www.ruby-forum.com/t/nginx-lua-and-blocking-libraries/238425/3 "2014-01-09T19:23:02Z")

</div>

Thanks a lot for the detailed answer, Yichun! I’ll try to benchmark it,  
estimate the db size, see if it fits in memory, etc.

Cheers,  
Andre