Sqlite on App Engine - Alternative to deleting ActiveRecord?

Ruby JRuby
1

In Ola B.'s instructions for JRuby-Rails on App Engine, he recommends
deleting ActiveRecord in order to meet the 1000-file limit.

What I’m wondering is, if I want to use ActiveRecord with a Sqlite
database
and bypass the Google Datastore, could I do this by deleting some of the
following services instead:

  • actionmailer

  • actionpack

  • activeresource

  • activesupport

Let’s assume I have a barebones scaffolded Rails app that just does some
basic CRUD. . .

Which of these services could I not do without?

2

I’m pretty sure you will definitely need ActionPack and ActiveSupport.
AP
does the controller and view work and AS is utility stuff used all
throughout rails. I think ActionMailer and ActiveResource could be
lived
without, though I don’t know how many files each of these parts
constitute.

Joe

3

ActionMailer and ActiveResource together only comes to a total of around
140
files. That would leave me still about 300 files over the limit. So it
looks
like ActiveRecord needs to go, as Ola suggested.

Thanks for the info.

4

Hi Ted

Thanks for the feedback. As you probably know there’s not much written
about
this approach but I’ve tried out Ola’s technique and I have found that
it
works if you delete ActiveRecord as he recommends. But I thought the
reason
it worked was because of the reduced file count. Even jar’d up I think
everything needs to be under 1000 files, but maybe you’re right - maybe
there’s something about ActiveRecord that doesn’t play well with App
Engine
even if you’re under the file limit. I’m soon going to find out :slight_smile:

What I just realized is that after deleting all of the test directories,
I’m
down to 855 files. That doesn’t leave much room for my app but it’s
enough
to develop a simple app without busting the limit. Now I just need to
try it
out and see if it works.

Chris

5

Hi Chris,

I don’t know whether it’s feasible to take the stuff in activerecord and
turn them into a jar. The way that Reggae and the appengine-apis are
set
up, is to freeze all your gems, stash them into a jar, and that
significantly reduces the number of files in your application. I
personally
am a Merb/DataMapper user, so i haven’t given a Rails setup a go really
(although i have followed along with what Ola’s been up to).

There has been some discussion of weird file path issues when jarring up
Rails, but again having not actually set that up, i don’t know
personally
yet.

That may not be helpful, but perhaps that gets you on the right path.

-Ted (knowtheory)

6

Hey Chris, Josh,

Josh is correct. The file limit is # of files on the app engine file
system, not the # in the jars.

-Ted

7

Hi Chris,
if you jar the rails gems it should work fine check out
JRuby 1.1.6: Gems-in-a-jar.
The thousand file limit, in my experience, is just the count of the
tiles
in the war directory. I had an app of over a thousand files and after I
jared up rails it works fine with no hiccups. So it should work for you
as
well.

Josh

8

Hi Josh,

This is great. Thanks for pointing out my misunderstanding.

Chris

9

Chris C. wrote:

In Ola B.'s instructions for JRuby-Rails on App Engine, he recommends
deleting ActiveRecord in order to meet the 1000-file limit.

What I’m wondering is, if I want to use ActiveRecord with a Sqlite
database and bypass the Google Datastore, could I do this by deleting
some of the following services instead:

I think you’re stuck right here…the problem is that all the JRuby
ActiveRecord backends are based on SQL APIs that are not available on
AppEngine. So I think you may be dead in the water with this approach.

I think your best bet is certainly going to be to go with DataMapper
atop Google’s persistence APIs.

  • Charlie

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email
10

Hi Charlie

Thanks for responding.

I’ve been flailing around for a few hours trying to get this to work
without
success. I was going to try it again tomorrow but what I’m hearing from
you
is that this goes beyond the “file count” limit and has to do with a
basic
incompatibility between ActiveRecord/Sqlite and App Engine. I was hoping
that I could use a bundled sqlite3 database and pretend that the Google
Datastore didn’t exist. Oh well.

In that case I’ll save the effort and use the recommended approach
instead.
When you say Datamapper, I assume you mean Genki Takiuchi’s
dm-datastore-adapter:

My understanding was that Datamapper itself won’t work with App Engine
until
version 0.10.

Thanks for setting me straight - this will save me from wasting a lot of
time.

Regards,
Chris

On Tue, Jun 2, 2009 at 10:41 PM, Charles Oliver N. <

11

Chris,

I’ve realized that unfortunately you’re not going to be able to use
sqlite
databases either. I’ve just remembered that AppEngine doesn’t allow you
to
write to the file system, so even if you do have an sqlite database, and
the
sqlite jdbc drivers or whatever, you won’t be able to alter your
database.

I’ve not tried out Genki’s adapter, although i’ve talked with him
briefly
about it once.

My strong preference and recommendation is to use Ryan B.'s
appengine-apis and his dm-appengine adapter (
Google Code Archive - Long-term storage for Google Code Project Hosting. they’re both available as gems
as
well currently named ‘appengine-apis’ and ‘dm-appengine’). Genki’s
adapter
is an 0.9.x adapter, and there are a number of changes that adapters
need to
go through to work with 0.10. Ryan’s adapter is an 0.10 adapter.
DataMapper 0.10 is very soon to drop (i’m hoping next week, but it
depends
on dkubb!)

Ryan, additionally, is a member of google’s appengine team, so i take
his
code to be a bit more on the authoritative side. The appengine-apis
provide
a thin ruby wrapper to the Java Appengine APIs in such a way that you
can
test against them, and run them on the local dev server and the like.
Ryan’s dm-appengine is an abstraction layer on top of his api wrapper,
which
means that you can additionally test your code locally w/ his appengine
adapter, and again, run them against the jetty dev server that google
provides in the sdk. As far as i am aware genki doesn’t provide any
promises that his adapter will run locally.

I should add that there’s more info/work on this coming down the pipe.
I’ve
been talking with a Ryan and another googler (John Woodell) and we’ve
got
some ideas as to how to make setup and deployment w/ appengine a little
bit
more ruby like. Only thing is, that i’m waiting on dm 0.10 and the
JRuby
Jars gems that we depend on to be released before i can write a tutorial
that can be used by anyone.

I personally use the appengine-apis gem and the dm-appengine adapter,
along
with a lib that i put together called reggae (Ruby Enabled GooGle App
Engine
http://github.com/knowtheory/reggae ). Reggae actually wraps all the
parts
of the java sdk that the appengine-apis need to use, and provides access
to
the jetty dev server, and (soon) the upload and app configuration tools
that
google includes in the sdk.

Eventually the goal is to have something as stupid simple as

merb-gen app your-app-name
reggae upload your-app-name
[enter your account and password]

(most likely there will be an “edit your config.ru to provide metadata”
step
inbetween those two steps)

Hope that helps!

-Ted (knowtheory)

12

Ted,

Thanks for the suggestion. I’ve been trying to get appengine-apis to
work
since reading your response.

Do you know if there is any documentation on setting Rails up with
appengine-apis and deploying to App Engine?

Only Merb seems to be documented on the wiki:
http://code.google.com/p/appengine-jruby/wiki/GettingStarted

I found this walk-through but couldn’t get it to work:
TargetX.
Perhaps it worked specifically for the sample app mentioned. Hasn’t
worked
with my skeleton Rails app so far. Or maybe some key instruction was
accidentally omitted from the writeup.

Hopefully Reggae will simplify this whole process. There are quite a lot
of
hoops to jump through at the moment :slight_smile:

Regards & Thanks,
Chris