Weird assignment problem, very confused :(

Rails
21

@Christoph, i’ll look at it late again if i have time: i still do not
understand what is the difference between this and my application, and
in my application it stores request.remote_ip as a normal string.
Maybe as a temporary workaround you can try to parse or convert
request.remote_ip back and forth between some formats a few times to
get it to be a normal string.

Alexey.

22

Alexey M. wrote in post #1019177:

I confirm that there is a problem.
This is with Rails 3.1.0rc8

$ rails new test_ip
$ cd test_ip
$ rails generate model MyModel ip:string description:string
$ rake db:migrate
$ rails generate controller Writer write_records

app/controllers/writer_controller.rb:

class WriterController < ApplicationController
def write_records
MyModel.create!(:ip => request.remote_ip, :description => ‘request’)
MyModel.create!(:ip => ‘127.0.0.1’, :description => ‘string’)
end
end

$ rails s

go to
http://localhost:3000/writer/write_records

$ rails c

MyModel.all
MyModel Load (0.2ms) SELECT “my_models”.* FROM “my_models”
=> [#<MyModel id: 1, ip: “127.0.0.1”, description: “request”,
created_at: “2011-08-30 09:42:09”, updated_at: “2011-08-30 09:42:09”>,
#<MyModel id: 2, ip: “127.0.0.1”, description: “string”, created_at:
“2011-08-30 09:42:09”, updated_at: “2011-08-30 09:42:09”>]

MyModel.where(:ip => ‘127.0.0.1’).all
MyModel Load (0.2ms) SELECT “my_models”.* FROM “my_models” WHERE
“my_models”.“ip” = ‘127.0.0.1’
=> [#<MyModel id: 2, ip: “127.0.0.1”, description: “string”,
created_at: “2011-08-30 09:42:09”, updated_at: “2011-08-30 09:42:09”>]

MyModel.first.ip == ‘127.0.0.1’
MyModel Load (0.2ms) SELECT “my_models”.* FROM “my_models” LIMIT 1
=> true

In the database for the first ip i have: X’3132372E302E302E31’

Yes, Christoph linked to his sqlite dump, which showed the same thing::

dump shows:

sqlite> .dump doc_types
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE “doc_types” (“id” INTEGER PRIMARY KEY AUTOINCREMENT NOT
NULL, “name” varchar(255), “description” varchar(255));
INSERT INTO “doc_types” VALUES(6,X’3132372E302E302E31’,‘request’);
INSERT INTO “doc_types” VALUES(7,‘127.0.0.1’,‘string’);

It sure seems like an encoding issue.

Also if you read the sqlite docs,
labeling a column as a certain type is not very rigid: you can insert
different types in a column. In fact, the sqlite2 docs say that having
typed columns is a “misfeature” of other databases.

sqlite3 docs:

SQLite supports the concept of “type affinity” on columns. The type
affinity of a column is the recommended type for data stored in that
column. The important idea here is that the type is recommended, not
required. Any column can still store any type of data. It is just that
some columns, given the choice, will prefer to use one storage class
over another. The preferred storage class for a column is called its
“affinity”.

In any case, just because the column type is varchar(255)
does not mean the data stored in the column has the same type–much less
the same encoding.

23

This is a weird issue!
@7stud, you should agree this is bug.
I have found out the difference between my application and this one.

In the controller, instead of

MyModel.create!(:ip => request.remote_ip, :description => 'request')

do

model = MyModel.new
model.save  # this is important!
model.ip = request.remote_ip
model.description = 'request'
model.save

then it works.

Alexey.

24

The encoding of the ip address as returned by remote_ip() is ASCII-8BIT,
and the encoding of the string literal ‘127.0.0.1’ in rails 3.0.9 is
US-ASCII. You can demonstrate that by putting this in an action:

@remote_ip = request.remote_ip
@remote_ip_encoding = @remote_ip.encoding.name
@str_literal_encoding = '127.0.0.1'.encoding.name

and then displaying the variables in a view. ASCII-8BIT is a synonym
for ‘binary’, i.e. unknown encoding. However, I can’t pin down how that
affects things because:

str = ‘abc’
str2 = ‘abc’
puts str.encoding.name
str.force_encoding(‘ASCII-8BIT’)
puts str.encoding.name

if str2 == str
puts ‘yes’
else
puts ‘no’
end

–output:–
US-ASCII
ASCII-8BIT
yes

25

7stud – wrote in post #1019185:

However, I can’t pin down how that
affects things because:

Actually, I don’t think it has anything to do with ruby–rather it’s how
sqlite3 compares the search string to the two strings in the sqlite3 db.

26

Alexey M. wrote in post #1019182:

This is a weird issue!
@7stud, you should agree this is bug.

I don’t know enough about rails to determine that. Encoding issues are
tricky.

I have found out the difference between my application and this one.

In the controller, instead of

MyModel.create!(:ip => request.remote_ip, :description => 'request')

do

model = MyModel.new
model.save  # this is important!
model.ip = request.remote_ip
model.description = 'request'
model.save

then it works.

Alexey.

I can’t duplicate the op’s problem: I can’t get sqlite3 to show
X’3132372E302E302E31’ for a string field. It doesn’t matter if I use
create(), or new() and save(). I get the following in both cases:

sqlite> .dump users
BEGIN TRANSACTION;
CREATE TABLE “users” (“id” INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
“name” varchar(255), “email” varchar(255), “created_at” datetime,
“updated_at” datetime, “last_sent_to” varchar(255));
INSERT INTO “users” VALUES(1,‘127.0.0.1’,‘request’,‘2011-08-30
11:43:08.702303’,‘2011-08-30 11:43:08.702303’,NULL);
INSERT INTO “users” VALUES(2,‘127.0.0.1’,‘string’,‘2011-08-30
11:43:08.710679’,‘2011-08-30 11:43:08.710679’,NULL);
COMMIT;
sqlite>

$ rails -v
Rails 3.0.9

$ ruby -v
ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.7.0]

$ sqlite3 -version
3.6.12

controller:

class PagesController < ApplicationController
def home
@title = “Home”
@remote_ip = request.remote_ip
@remote_ip_encoding = @remote_ip.encoding.name
@str_literal_encoding = ‘127.0.0.1’.encoding.name

User.create!(:name => request.remote_ip, :email => 'request')
User.create!(:name => '127.0.0.1', :email => 'string')

=begin
model.save # this is important!
model.name = request.remote_ip
model.email = ‘request’
model.save

model2 = User.new
model2.save
model2.name = '127.0.0.1'
model2.email = 'string'
modelT.save

User.find_by_name('127.0.0.1')

=end

end

def about
@title = “About”
end

end

27

I ran my action like this too:

class PagesController < ApplicationController
def home
@title = “Home”
@remote_ip = request.remote_ip
@remote_ip_encoding = @remote_ip.encoding.name
@str_literal_encoding = ‘127.0.0.1’.encoding.name

User.create!(:name => request.remote_ip, :email => 'request')
User.create!(:name => '127.0.0.1', :email => 'string')

model = User.new
model.save  # this is important!
model.name = request.remote_ip
model.email = 'request'
model.save

model2 = User.new
model2.save
model2.name = '127.0.0.1'
model2.email = 'string'
model2.save

end

def about
@title = “About”
end

end

sqlite> .dump users
BEGIN TRANSACTION;
CREATE TABLE “users” (“id” INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
“name” varchar(255), “email” varchar(255), “created_at” datetime,
“updated_at” datetime, “last_sent_to” varchar(255));
INSERT INTO “users” VALUES(1,‘127.0.0.1’,‘request’,‘2011-08-30
12:04:18.230500’,‘2011-08-30 12:04:18.230500’,NULL);
INSERT INTO “users” VALUES(2,‘127.0.0.1’,‘string’,‘2011-08-30
12:04:18.237719’,‘2011-08-30 12:04:18.237719’,NULL);
INSERT INTO “users” VALUES(3,‘127.0.0.1’,‘request’,‘2011-08-30
12:04:18.302252’,‘2011-08-30 12:04:18.302252’,NULL);
INSERT INTO “users” VALUES(4,‘127.0.0.1’,‘string’,‘2011-08-30
12:04:18.307215’,‘2011-08-30 12:04:18.307215’,NULL);
COMMIT;
sqlite>

28

Thanks, 7stud, the issue is solved with

MyModel.create!(:ip => request.remote_ip.force_encoding(‘UTF-8’))

Apparently the sqlite adapter looks into the encoding before saving.
Still not clear if this should not be considered a bug.

Alexey.

29

7stud – wrote in post #1019190:

I can’t duplicate the op’s problem: I can’t get sqlite3 to show
X’3132372E302E302E31’ for a string field. It doesn’t matter if I use
create(), or new() and save().

$ rails -v
Rails 3.0.9

$ ruby -v
ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.7.0]

$ sqlite3 -version
3.6.12

Strange, maybe it is a difference between Rails 3.0.X and Rails 3.1?

30

Alexey M. wrote in post #1019192:

Thanks, 7stud, the issue is solved with

MyModel.create!(:ip => request.remote_ip.force_encoding(‘UTF-8’))

Apparently the sqlite adapter looks into the encoding before saving.

When I do a bundle install, this is the only sqlite3 gem I see:

Using sqlite3 (1.3.3)

What version are you guys using? Also what version of ruby, rails and
sqlite? Mine are:

$ rails -v
Rails 3.0.9

$ ruby -v
ruby 1.9.2p180 (2011-02-18 revision 30909) [x86_64-darwin10.7.0]

$ sqlite3 -version
3.6.12

31

7stud – wrote in post #1019196:

What version are you guys using? Also what version of ruby, rails and
sqlite?

$ rails -v
Rails 3.1.0.rc8

$ ruby -v
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin10.8.0]

$ sqlite3 -version
3.7.7.1 2011-06-28 17:39:05

32

Alexey M. wrote in post #1019202:

7stud – wrote in post #1019196:

What version are you guys using? Also what version of ruby, rails and
sqlite?

$ rails -v
Rails 3.1.0.rc8

$ ruby -v
ruby 1.9.2p290 (2011-07-09 revision 32553) [x86_64-darwin10.8.0]

$ sqlite3 -version
3.7.7.1 2011-06-28 17:39:05

Alexey,

I faced with the similar problem in slightly different environment:

(in simply controller)
…
hash = Digest::SHA2.hexdigest
[params[:document][:file].original_filename, current_user.email,
version].join
…
if current_user.documents.create :path => file.to_s, :hashref =>
hash, :version => version,
…

(logs and dump)
…
INSERT INTO “documents” (“created_at”, “description”, “hashref”,
“mimetype”, “path”, “updated_at”, “user_id”, “version”) VALUES (?, ?, ?,
?, ?, ?, ?, ?) [[“created_at”, Tue, 13 Sep 2011 17:43:16 UTC +00:00],
[“description”, nil],

[“hashref”, “18da91cbfadb74e43d9ca4231555462d”],
[“mimetype”, “[text/plain]”], [“path”, “y1.txt”], [“updated_at”, Tue, 13
Sep 2011 17:43:16 UTC +00:00], [“user_id”, 1], [“version”, 0]]

sqlite> .dump documents
INSERT INTO “documents”
VALUES(13,1,‘y1.txt’,

X’3138646139316362666164623734653433643963613432333135353534363264’,
NULL,’[text/plain]’,‘2011-09-13
17:43:16.214665’,‘2011-09-13 17:43:16.214665’,0);

So I can’t refer to this record using hash value. But! If I add just a
little modification to hash string, just a single char other than
[0-9a-f] (say hash += “*”), everything works well:

sqlite> .dump documents
INSERT INTO “documents”
VALUES(14,1,‘test-text’,

‘6c9034ddb7f1f248e6a215a1d335ff94*’,
‘tt1’,’[]’,‘2011-09-13
17:59:25.925458’,‘2011-09-13 17:59:25.925458’,0);

I’m using Rails 3.1, Ruby 1.9.2p290 and sqlite 3.7.7.1.

33

This is resolved in rails 3.1.3. At least i tried and request.remote_ip
was stored correctly without need to enforce utf-8 encoding.