Sqlite db adding garbage

Hello guys
After figuring out a way to load my db with info scraped of from a web
page, the data on db is appended with “— \n -” these characters. I’m
not sure what the problem is… I did change my editor to xcode.

I have attached my app’s

  • migration file
  • the method that is called to populate the database
  • the output on rails server

You can see I’m specifically printing out the values like (eg:
[item[0]]) before assigning it to dts.bug_id or any other db row for
that matter. I can see the printed value is fine but after storing on
the database they are being appended with those junk characters.

Have anyone of you noticed this before. Am I doing something
wrong…please help !!
Thanks in advance – Anush

class CreateBugs < ActiveRecord::Migration
def self.up
create_table :bugs do |t|
t.string :bug_id
t.string :bug_pri
t.string :bug_sev
t.string :bug_state
t.string :bug_date
t.string :bug_org
t.string :bug_asg
t.string :bug_sum
t.string :bug_fcomment
t.string :bug_fdate
t.timestamps
end
end

def self.down
drop_table :bugs
end
end

def scrap_dts
puts “Inside def scrap_dts in import controller”
agent = Mechanize.new
agent.basic_auth(‘anushj’, ‘porfavor’)
page =
agent.get(‘http://hidden/query.cgi?action=search&name=anushj&origin=me&priority=1&report_type=0&severity=1&subreport_id=46’)
table = agent.page.search(’//*[(@id =
“subreport_list”)]//td’).map(&:text).map(&:strip)

table = table.split('Bug')
table.delete_if{|x| x.size == 0}

table.each do |item|
  tmp = [item[0]]
  tmp = tmp.split('-')
  if Bug.find_by_bug_id(tmp[0]).nil?
    dts = Bug.new
    dts.bug_id = tmp[0]
    puts tmp[0]
    dts.bug_pri = [item[2]]
    puts [item[2]]
    dts.bug_sev = [item[4]]
    puts [item[4]]
    dts.bug_state = [item[5]]
    puts [item[5]]
    dts.bug_date = [item[6]]
    puts [item[6]]
    dts.bug_org = [item[9]]
    puts [item[9]]
    dts.bug_asg = [item[10]]
    puts [item[10]]
    dts.save
    puts "save record!"
    #dts_num = [item[0]]
    #puts dts_num
  end
end

end

DB O/P while executing scrap_dts

Inside def scrap_dts in import controller
106083-M30S
P1
S1
New
2011/02/16
anushj
mmourier
save record!
115412-M30S
P1
S1
Closed: Verified
2011/04/05
anushj
anushj
save record!
115762-M30S
P1
S1
Closed: Verified
2011/04/11
anushj
anushj
save record!

Started GET “/” for 127.0.0.1 at 2011-04-19 23:37:33 -0700
Processing by ImportController#index as HTML
Bug Load (0.3ms) SELECT “bugs”.* FROM “bugs” WHERE “bugs”.“bug_id” IN
(‘106083-M30S’) LIMIT 1
Bug Load (0.2ms) SELECT “bugs”.“id” FROM “bugs” WHERE
(“bugs”.“bug_id” = ‘[“106083-M30S”]’) LIMIT 1
AREL (0.4ms) INSERT INTO “bugs” (“bug_id”, “bug_pri”, “bug_sev”,
“bug_state”, “bug_date”, “bug_org”, “bug_asg”, “bug_sum”,
“bug_fcomment”, “bug_fdate”, “created_at”, “updated_at”) VALUES (’—

  • 106083-M30S
    ', '—
  • P1
    ', '—
  • S1
    ', '—
  • New
    ', '—
  • 2011/02/16
    ', '—
  • anushj
    ', '—
  • mmourier
    ‘, NULL, NULL, NULL, ‘2011-04-20 06:37:35.112929’, ‘2011-04-20
    06:37:35.112929’)
    Bug Load (0.3ms) SELECT “bugs”.* FROM “bugs” WHERE “bugs”.“bug_id” IN
    (‘115412-M30S’) LIMIT 1
    Bug Load (0.1ms) SELECT “bugs”.“id” FROM “bugs” WHERE
    (“bugs”.“bug_id” = ‘[“115412-M30S”]’) LIMIT 1
    AREL (0.3ms) INSERT INTO “bugs” (“bug_id”, “bug_pri”, “bug_sev”,
    “bug_state”, “bug_date”, “bug_org”, “bug_asg”, “bug_sum”,
    “bug_fcomment”, “bug_fdate”, “created_at”, “updated_at”) VALUES (’—
  • 115412-M30S
    ', '—
  • P1
    ', '—
  • S1
    ', '—
  • “Closed: Verified”
    ', '—
  • 2011/04/05
    ', '—
  • anushj
    ', '—
  • anushj
    ‘, NULL, NULL, NULL, ‘2011-04-20 06:37:35.124746’, ‘2011-04-20
    06:37:35.124746’)
    Bug Load (0.2ms) SELECT “bugs”.* FROM “bugs” WHERE “bugs”.“bug_id” IN
    (‘115762-M30S’) LIMIT 1
    Bug Load (0.1ms) SELECT “bugs”.“id” FROM “bugs” WHERE
    (“bugs”.“bug_id” = ‘[“115762-M30S”]’) LIMIT 1
    AREL (0.4ms) INSERT INTO “bugs” (“bug_id”, “bug_pri”, “bug_sev”,
    “bug_state”, “bug_date”, “bug_org”, “bug_asg”, “bug_sum”,
    “bug_fcomment”, “bug_fdate”, “created_at”, “updated_at”) VALUES (’—
  • 115762-M30S
    ', '—
  • P1
    ', '—
  • S1
    ', '—
  • “Closed: Verified”
    ', '—
  • 2011/04/11
    ', '—
  • anushj
    ', '—
  • anushj
    ', NULL, NULL, NULL, ‘2011-04-20 06:37:35.129591’, ‘2011-04-20
    06:37:35.129591’)
    Bug Load (3.9ms) SELECT “bugs”.* FROM “bugs”
    SQL (0.2ms) SELECT COUNT(*) FROM “bugs”
    Rendered import/index.html.erb within layouts/application (46.3ms)
    Completed 200 OK in 1769ms (Views: 51.0ms | ActiveRecord: 6.2ms)

On Apr 20, 9:18am, “Anush J.” [email protected] wrote:

You can see I’m specifically printing out the values like (eg:
[item[0]]) before assigning it to dts.bug_id or any other db row for
that matter. I can see the printed value is fine but after storing on
the database they are being appended with those junk characters.

Have anyone of you noticed this before. Am I doing something
wrong…please help !!

You’ve got a bunch of string columns but you’re assigning arrays to
them (such as [item[0]]), which forces activerecord to encode them as
yaml which is what causes the extra characters you are seeing - that
how yaml encodes arrays. If you did want to do this you should use the
serialize directive to inform activerecord that those columns should
be serialized/deserialized. If not, then just assign item[0] rather
than [item[0]]

Fred

Fred

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs