Forum: Ruby Localmemcache-0.2.0: The beauty of memcached. For local data. Blazingly fast.

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Sven C. Koehler (Guest)
on 2009-03-31 03:20
(Received via mailing list)
Hi,

I am happy to announce the next version of localmemcache (0.2.0), a
library that aims to provide an interface similar to memcached but for
accessing local data instead of remote data.  It's based on mmap()'ed
shared memory for maximum speed.

* http://localmemcache.rubyforge.org/

Changes for 0.2.0:
  * Localmemcache uses logging now: In case your application is
terminated
    while accessing the shared memory (eg by a kill -9), localmemcache
is
    now able to restore the integrity of your data.
  * \0 character can now be used in keys and values
  * The ruby binding now features a keys() method.
  * Added a C API.

0.2.0 facts:
  - lightweight: The core library is just about 1300 lines of C code
  - fast: About 40% slower than Ruby's hash (or about 36 times
    faster than a local memcached).

EXAMPLE
=======

require 'localmemcache'
$lm = LocalMemCache.new :namespace => "viewcounters"
$lm[:foo] = 1
$lm[:foo]
$lm.delete(:foo)

INSTALL
=======

# gem install localmemcache

(In case rubyforge has not yet updated the mirrors, fetch the 0.2.0 gem
from here: http://github.com/sck/localmemcache/downloads and then do
# gem install localmemcache-0.2.0.gem )

CONTACT
=======

Please contact me with bugs, suggestions and patches at: schween + snafu
# de

LINKS
=====

Localmemcache: http://localmemcache.rubyforge.org/
Rubyforge project: http://localmemcache.rubyforge.org/

Source code is hosted on github: http://github.com/sck/localmemcache/

Best,

Sven C. Koehler

afa5fe7a3a23930a50d0b8b4a1a55880  localmemcache-0.2.0.gem
050cab44c9a5b9cb6214779398c0cfb7  localmemcache-0.2.0.tar.gz
Ara H. (Guest)
on 2009-03-31 06:28
(Received via mailing list)
On Mar 30, 2009, at 5:17 PM, Sven C. Koehler wrote:

> # gem install localmemcache

really nice idea.  can't wait to try it out!

a @ http://codeforpeople.com/
Bill K. (Guest)
on 2009-03-31 11:14
(Received via mailing list)
From: "Sven C. Koehler" <removed_email_address@domain.invalid>
>
> I am happy to announce the next version of localmemcache (0.2.0), a
> library that aims to provide an interface similar to memcached but for
> accessing local data instead of remote data.  It's based on mmap()'ed
> shared memory for maximum speed.

Very nice!!

I've been daydreaming about a mmap'd hash table for a
couple years now -- and I wasn't even contemplating
having it able to be shared between processes, as it
appears you've done.

Your code appears nicely decoupled.  It seems possible
if one were to write Windoze versions of lmc_lock.c and
lmc_shm.c , the rest of the code might just work on that
platform.  (I may attempt this... someday...)


Anyway, thanks for releasing this interesting library!


Regards,

Bill
Sven C. Koehler (Guest)
on 2009-03-31 18:33
(Received via mailing list)
On Tue, Mar 31, 2009 at 04:13:39PM +0900, Bill K. wrote:
> From: "Sven C. Koehler" <removed_email_address@domain.invalid>
>>
>> I am happy to announce the next version of localmemcache (0.2.0), a
>> library that aims to provide an interface similar to memcached but for
>> accessing local data instead of remote data.  It's based on mmap()'ed
>> shared memory for maximum speed.
>
> Very nice!!

Thanks!

> Your code appears nicely decoupled.  It seems possible
> if one were to write Windoze versions of lmc_lock.c and
> lmc_shm.c , the rest of the code might just work on that
> platform.  (I may attempt this... someday...)

Yeah, this should work. ;-)

-S.
Michael F. (Guest)
on 2009-04-04 20:41
(Received via mailing list)
On Tue, 31 Mar 2009 08:17:45 +0900
"Sven C. Koehler" <removed_email_address@domain.invalid> wrote:

> Hi,
>
> I am happy to announce the next version of localmemcache (0.2.0), a
> library that aims to provide an interface similar to memcached but for
> accessing local data instead of remote data.  It's based on mmap()'ed
> shared memory for maximum speed.

This sounds great, can't wait to add it as a caching option to Ramaze.


> 0.2.0 facts:
> $lm[:foo]
> $lm.delete(:foo)

I tried that, but $lm[:foo] always returned nil.
That's on ArchLinux, both 32 and 64bit.
Radosław B. (Guest)
on 2009-04-04 20:53
(Received via mailing list)
On Sat, Apr 4, 2009 at 6:36 PM, Michael F.
<removed_email_address@domain.invalid> wrote:
>> require 'localmemcache'
>> $lm = LocalMemCache.new :namespace => "viewcounters"
>> $lm[:foo] = 1
>> $lm[:foo]
>> $lm.delete(:foo)
>
> I tried that, but $lm[:foo] always returned nil.
> That's on ArchLinux, both 32 and 64bit.
>

The same for me. My configuration: ubuntu 8.10 64bit, ruby 1.8.7.

--
Pozdrawiam

Rados³aw Bu³at
http://radarek.jogger.pl - mój blog
Sven C. Koehler (Guest)
on 2009-04-04 21:51
(Received via mailing list)
On Sun, Apr 05, 2009 at 01:36:09AM +0900, Michael F. wrote:
> On Tue, 31 Mar 2009 08:17:45 +0900
> "Sven C. Koehler" <removed_email_address@domain.invalid> wrote:
> > require 'localmemcache'
> > $lm = LocalMemCache.new :namespace => "viewcounters"
> > $lm[:foo] = 1
> > $lm[:foo]
> > $lm.delete(:foo)
>
> I tried that, but $lm[:foo] always returned nil.
> That's on ArchLinux, both 32 and 64bit.

Hmm... Can you see the namespace in /var/tmp/localmemcache/ ?
Can you see the key in $lm.keys()?
Does ``du -h /var/tmp/localmemcache/viewcounters.lmc'' show that there's
content in the file?

Best,

Sven
Sven C. Koehler (Guest)
on 2009-04-04 22:01
(Received via mailing list)
On Sat, Apr 04, 2009 at 07:51:07PM +0200, Sven C. Koehler wrote:
> Hmm... Can you see the namespace in /var/tmp/localmemcache/ ?
> Can you see the key in $lm.keys()?
> Does ``du -h /var/tmp/localmemcache/viewcounters.lmc'' show that there's
> content in the file?

This is how it looks on my machine:

| sck@comp ~/tmp $ uname -a
| Linux comp 2.6.23-gentoo-r3-b #6 SMP Sat Aug 23 08:57:32 UTC 2008
| x86_64 Intel(R) Xeon(R) CPU E5205 @ 1.86GHz GenuineIntel GNU/Linux
| sck@comp ~/tmp $ irb
| irb(main):001:0> require 'localmemcache'
| => true
| irb(main):002:0> $lm = LocalMemCache.new :namespace => "viewcounters"
| => #<LocalMemCache:0x2b5ed7ef0130>
| irb(main):003:0> $lm.keys
| => []
| irb(main):004:0> $lm[:foo]
| => nil
| irb(main):005:0> $lm[:foo] = 1
| => 1
| irb(main):006:0> $lm[:foo]
| => "1"
| irb(main):007:0> $lm.keys
| => ["foo"]
| irb(main):008:0> sck@comp ~/tmp $
| sck@comp ~/tmp $ ls -al /var/tmp/localmemcache/viewcounters.lmc
| -rwxr-xr-x 1 sck sck 1073741824 Apr  4 17:52
|     /var/tmp/localmemcache/viewcounters.lmc
| sck@comp ~/tmp $ du -h /var/tmp/localmemcache/viewcounters.lmc
| 16K     /var/tmp/localmemcache/viewcounters.lmc
| sck@comp ~/tmp $ irb
| irb(main):002:0> require 'localmemcache'
| => true
| irb(main):003:0> $lm = LocalMemCache.new :namespace => "viewcounters"
| => #<LocalMemCache:0x2b50be3e6978>
| irb(main):004:0> $lm.keys
| => ["foo"]
| irb(main):005:0>

-S.
Radosław B. (Guest)
on 2009-04-05 01:58
(Received via mailing list)
On Sat, Apr 4, 2009 at 7:50 PM, Sven C. Koehler 
<removed_email_address@domain.invalid>
wrote:
> Hmm... Can you see the namespace in /var/tmp/localmemcache/ ?

Yes
$ ls /var/tmp/localmemcache/
foo.lmc  viewcounters.lmc


> Can you see the key in $lm.keys()?
Yes, but they are duplicated after setting on the same key:

>> $lm = LocalMemCache.new :namespace => "viewcounters"
=> #<LocalMemCache:0x7f9c05d844d0>
>> $lm.keys
=> []
>> $lm[:foo] = 1
=> 1
>> $lm[:foo]
=> nil
>> $lm.keys
=> ["foo"]
>> $lm[:foo] = 2
=> 2
>> $lm[:foo]
=> nil
>> $lm.keys
=> ["foo", "foo"]

> Does ``du -h /var/tmp/localmemcache/viewcounters.lmc'' show that there's
> content in the file?
>

$ du -h /var/tmp/localmemcache/viewcounters.lmc
20K  /var/tmp/localmemcache/viewcounters.lmc


--
Pozdrawiam

Rados³aw Bu³at
http://radarek.jogger.pl - mój blog
Matthias R. (Guest)
on 2009-04-05 02:12
(Received via mailing list)
On Sat, 04 Apr 2009 12:59:56 -0500, Sven C. Koehler wrote:
> This is how it looks on my machine:

I can reproduce the issue Michael and Radosław are having:

  reima@marvin:~$ uname -a
  Linux marvin 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009
i686
  GNU/Linux
  reima@marvin:~$ ruby -v
  ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
  reima@marvin:~$ irb
  >> require 'localmemcache'
  => true
  >> $lm = LocalMemCache.new :namespace => "viewcounters"
  => #<LocalMemCache:0xb7b69bdc>
  >> $lm.keys
  => []
  >> $lm[:foo]
  => nil
  >> $lm[:foo] = 1
  => 1
  >> $lm[:foo]
  => nil
  >> $lm.keys
  => ["foo"]
  >> puts `du -h /var/tmp/localmemcache/viewcounters.lmc`
  16K  /var/tmp/localmemcache/viewcounters.lmc
  => nil
  >>

And on a sidenote:

  reima@marvin:~$ irb
  >> require 'localmemcache'
  => true
  >> $lm = LocalMemCache.new :namespace => "viewcounters"
  => #<LocalMemCache:0xb79eac20>
  >> $lm.close
  => nil
  >> $lm.keys
  (irb):4: [BUG] Segmentation fault
  ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]

  Aborted
  reima@marvin:~$

-Matthias
Sven C. Koehler (Guest)
on 2009-04-05 17:10
(Received via mailing list)
Ok, thanks to everyone who reported bugs!
I think I fixed this problem, so could anyone who had problems setting
keys run the following command to verify whether it works better now?

% git clone git://github.com/sck/localmemcache.git && \
     cd localmemcache && rake sanity_test

The output at the end should look like this:

| LocalMemCache
| - should allow to set and query keys
| - should support the [] and []= operators
| - should allow deletion of keys
| - should return a list of keys
| - should support \0 in values and keys
| - should throw exception if pool is full
| - should support checking of namespaces
| - should support clearing of namespaces
|
| 8 specifications (8 requirements), 0 failures, 0 errors

(I haven't fixed the close bug yet.)

Best,

Sven
Radosław B. (Guest)
on 2009-04-05 17:21
(Received via mailing list)
On Sun, Apr 5, 2009 at 3:09 PM, Sven C. Koehler 
<removed_email_address@domain.invalid>
wrote:
> | - should allow to set and query keys
> | - should support the [] and []= operators
> | - should allow deletion of keys
> | - should return a list of keys
> | - should support \0 in values and keys
> | - should throw exception if pool is full
> | - should support checking of namespaces
> | - should support clearing of namespaces
> |
> | 8 specifications (8 requirements), 0 failures, 0 errors

It works as you expected:

LocalMemCache
- should allow to set and query keys
- should support the [] and []= operators
- should allow deletion of keys
- should return a list of keys
- should support \0 in values and keys
- should throw exception if pool is full
- should support checking of namespaces
- should support clearing of namespaces

8 specifications (8 requirements), 0 failures, 0 errors
--
Pozdrawiam

Rados³aw Bu³at
http://radarek.jogger.pl - mój blog
Michael F. (Guest)
on 2009-04-05 18:22
(Received via mailing list)
On Sun, 5 Apr 2009 22:09:04 +0900
"Sven C. Koehler" <removed_email_address@domain.invalid> wrote:

> | - should allow to set and query keys
> | - should support the [] and []= operators
> | - should allow deletion of keys
> | - should return a list of keys
> | - should support \0 in values and keys
> | - should throw exception if pool is full
> | - should support checking of namespaces
> | - should support clearing of namespaces
> |
> | 8 specifications (8 requirements), 0 failures, 0 errors

Aye, now it works as expected, thanks a lot. Will we get an updated
release as
well?
This topic is locked and can not be replied to.