Hi all, I have a Rails 3.2.12 app running under JRuby 1.7.2. In working with a particular payment gateway, I have a user who is entering a name with a French accented character as her user name (e.g. Labbé). When the gateway posts back to my Rails app to update the user's profile, I'm getting the following: An ArgumentError occurred in users#update_profile: invalid byte sequence in UTF-8 org/jruby/RubyString.java:3074:in `gsub' I can see the parameters being passed to my app as: Parameters: {"data_key"=>"(Edited)", "cust_id"=>"316", "date_stamp"=>"2013-09-06", "time_stamp"=>"09:12:08", "response_code"=>"001", "message"=>"Successfully registered CC details.", "res_success"=>"true", "payment_type"=>"cc", "cardholder"=>"K Labb\xE9", "expiry_date"=>"1312", "f4l4"=>"4242***4242", "result"=>"1", "avs_response_code"=>"", "cvd_response_code"=>"", "controller"=>"users", "action"=>"update_profile"} In my app, I'm recording the name provided as follows: user = User.find(user_id) if user.present? #Record the name as it was entered on the Hosted Vault user.name_on_card = params[:cardholder] Page user.datakey = params[:data_key] #This is the data key used for subsequent transactions .... I already have my app configured for UTF-8 (in environment.rb) Encoding.default_external = Encoding::UTF_8 Encoding.default_internal = Encoding::UTF_8 How do I properly record this user's name without triggering the "invalid byte sequence" error? Many thanks! Tim For reference, the stack trace: ------------------------------- Backtrace: ------------------------------- org/jruby/RubyString.java:3074:in `gsub' org/jruby/RubyString.java:3042:in `gsub' vendor/cache/activerecord-jdbc-adapter-91d319a99c18/lib/arjdbc/mssql/adapter.rb:221:in `quote_string' vendor/cache/activerecord-jdbc-adapter-91d319a99c18/lib/arjdbc/mssql/adapter.rb:210:in `quote' arel (3.0.2) lib/arel/visitors/to_sql.rb:416:in `quote' arel (3.0.2) lib/arel/visitors/to_sql.rb:331:in `visit_Arel_Nodes_Assignment' org/jruby/RubyBasicObject.java:1665:in `__send__' org/jruby/RubyKernel.java:2090:in `send' arel (3.0.2) lib/arel/visitors/visitor.rb:19:in `visit' arel (3.0.2) lib/arel/visitors/to_sql.rb:61:in `visit_Arel_Nodes_UpdateStatement' org/jruby/RubyArray.java:2361:in `map' arel (3.0.2) lib/arel/visitors/to_sql.rb:61:in `visit_Arel_Nodes_UpdateStatement' org/jruby/RubyBasicObject.java:1665:in `__send__' org/jruby/RubyKernel.java:2090:in `send' arel (3.0.2) lib/arel/visitors/visitor.rb:19:in `visit' arel (3.0.2) lib/arel/visitors/visitor.rb:5:in `accept' arel (3.0.2) lib/arel/visitors/to_sql.rb:19:in `accept' activerecord (3.2.13) lib/active_record/connection_adapters/abstract/database_statements.rb:7:in `to_sql' activerecord (3.2.13) lib/active_record/connection_adapters/abstract/database_statements.rb:96:in `update' activerecord (3.2.13) lib/active_record/connection_adapters/abstract/query_cache.rb:19:in `update' activerecord (3.2.13) lib/active_record/persistence.rb:359:in `update' activerecord (3.2.13) lib/active_record/locking/optimistic.rb:68:in `update' activerecord (3.2.13) lib/active_record/attribute_methods/dirty.rb:74:in `update' activerecord (3.2.13) lib/active_record/timestamp.rb:71:in `update' activerecord (3.2.13) lib/active_record/callbacks.rb:272:in `update' activesupport (3.2.13) lib/active_support/callbacks.rb:406:in `_run__1472338488__update__1343517959__callbacks' org/jruby/RubyBasicObject.java:1659:in `__send__' org/jruby/RubyKernel.java:2086:in `send' activesupport (3.2.13) lib/active_support/callbacks.rb:405:in `__run_callback' activesupport (3.2.13) lib/active_support/callbacks.rb:390:in `_run_update_callbacks' org/jruby/RubyBasicObject.java:1659:in `__send__' org/jruby/RubyKernel.java:2086:in `send' activesupport (3.2.13) lib/active_support/callbacks.rb:81:in `run_callbacks' activerecord (3.2.13) lib/active_record/callbacks.rb:272:in `update' vendor/cache/hobo-9dacbc4465df/hobo/lib/hobo/model/permissions.rb:146:in `update_with_hobo_permission_check' activerecord (3.2.13) lib/active_record/persistence.rb:348:in `create_or_update' activerecord (3.2.13) lib/active_record/callbacks.rb:264:in `create_or_update' activesupport (3.2.13) lib/active_support/callbacks.rb:439:in `_run__1472338488__save__1343517959__callbacks' org/jruby/RubyBasicObject.java:1659:in `__send__' org/jruby/RubyKernel.java:2086:in `send' activesupport (3.2.13) lib/active_support/callbacks.rb:405:in `__run_callback' activesupport (3.2.13) lib/active_support/callbacks.rb:390:in `_run_save_callbacks' org/jruby/RubyBasicObject.java:1659:in `__send__' org/jruby/RubyKernel.java:2086:in `send' activesupport (3.2.13) lib/active_support/callbacks.rb:81:in `run_callbacks' activerecord (3.2.13) lib/active_record/callbacks.rb:264:in `create_or_update' activerecord (3.2.13) lib/active_record/persistence.rb:104:in `save!' activerecord (3.2.13) lib/active_record/validations.rb:56:in `save!' activerecord (3.2.13) lib/active_record/attribute_methods/dirty.rb:33:in `save!' activerecord (3.2.13) lib/active_record/transactions.rb:264:in `save!' activerecord (3.2.13) lib/active_record/transactions.rb:313:in `with_transaction_returning_status' activerecord (3.2.13) lib/active_record/connection_adapters/abstract/database_statements.rb:192:in `transaction' torquebox-transactions (2.3.0) lib/torquebox/active_record_adapters.rb:38:in `transaction' activerecord (3.2.13) lib/active_record/transactions.rb:208:in `transaction' activerecord (3.2.13) lib/active_record/transactions.rb:311:in `with_transaction_returning_status' activerecord (3.2.13) lib/active_record/transactions.rb:264:in `save!' app/controllers/users_controller.rb:202:in `update_profile' org/jruby/RubyBasicObject.java:1659:in `__send__' org/jruby/RubyKernel.java:2086:in `send' actionpack (3.2.13) lib/action_controller/metal/implicit_render.rb:4:in `send_action' actionpack (3.2.13) lib/abstract_controller/base.rb:167:in `process_action' actionpack (3.2.13) lib/action_controller/metal/rendering.rb:10:in `process_action' actionpack (3.2.13) lib/abstract_controller/callbacks.rb:18:in `process_action' activesupport (3.2.13) lib/active_support/callbacks.rb:461:in `_run__260237797__process_action__885198886__callbacks' org/jruby/RubyBasicObject.java:1659:in `__send__' org/jruby/RubyKernel.java:2086:in `send' activesupport (3.2.13) lib/active_support/callbacks.rb:405:in `__run_callback' activesupport (3.2.13) lib/active_support/callbacks.rb:390:in `_run_process_action_callbacks' org/jruby/RubyBasicObject.java:1665:in `__send__' org/jruby/RubyKernel.java:2090:in `send' activesupport (3.2.13) lib/active_support/callbacks.rb:81:in `run_callbacks' actionpack (3.2.13) lib/abstract_controller/callbacks.rb:17:in `process_action' actionpack (3.2.13) lib/action_controller/metal/rescue.rb:29:in `process_action' actionpack (3.2.13) lib/action_controller/metal/instrumentation.rb:30:in `process_action' activesupport (3.2.13) lib/active_support/notifications.rb:123:in `instrument' activesupport (3.2.13) lib/active_support/notifications/instrumenter.rb:20:in `instrument' activesupport (3.2.13) lib/active_support/notifications/instrumenter.rb:19:in `instrument' activesupport (3.2.13) lib/active_support/notifications.rb:123:in `instrument' actionpack (3.2.13) lib/action_controller/metal/instrumentation.rb:29:in `process_action' actionpack (3.2.13) lib/action_controller/metal/params_wrapper.rb:207:in `process_action' activerecord (3.2.13) lib/active_record/railties/controller_runtime.rb:18:in `process_action' actionpack (3.2.13) lib/abstract_controller/base.rb:121:in `process' actionpack (3.2.13) lib/abstract_controller/rendering.rb:45:in `process' actionpack (3.2.13) lib/action_controller/metal.rb:203:in `dispatch' actionpack (3.2.13) lib/action_controller/metal/rack_delegation.rb:14:in `dispatch' actionpack (3.2.13) lib/action_controller/metal.rb:246:in `action' org/jruby/RubyProc.java:249:in `call' actionpack (3.2.13) lib/action_dispatch/routing/route_set.rb:73:in `dispatch' actionpack (3.2.13) lib/action_dispatch/routing/route_set.rb:36:in `call' journey (1.0.4) lib/journey/router.rb:68:in `call' org/jruby/RubyArray.java:1613:in `each' journey (1.0.4) lib/journey/router.rb:56:in `call' actionpack (3.2.13) lib/action_dispatch/routing/route_set.rb:612:in `call' exception_notification (3.0.1) lib/exception_notifier.rb:41:in `call' actionpack (3.2.13) lib/action_dispatch/middleware/best_standards_support.rb:17:in `call' rack (1.4.5) lib/rack/etag.rb:23:in `call' rack (1.4.5) lib/rack/conditionalget.rb:35:in `call' actionpack (3.2.13) lib/action_dispatch/middleware/head.rb:14:in `call' actionpack (3.2.13) lib/action_dispatch/middleware/params_parser.rb:21:in `call' actionpack (3.2.13) lib/action_dispatch/middleware/flash.rb:242:in `call' torquebox-web-2.3.0 (java) lib/torquebox/session/servlet_store.rb:31:in `call' actionpack (3.2.13) lib/action_dispatch/middleware/cookies.rb:341:in `call' activerecord (3.2.13) lib/active_record/query_cache.rb:64:in `call' activerecord (3.2.13) lib/active_record/connection_adapters/abstract/connection_pool.rb:479:in `call' actionpack (3.2.13) lib/action_dispatch/middleware/callbacks.rb:28:in `call' activesupport (3.2.13) lib/active_support/callbacks.rb:408:in `_run__ 2063131183__call__1343517959__callbacks' org/jruby/RubyBasicObject.java:1659:in `__send__' org/jruby/RubyKernel.java:2086:in `send' activesupport (3.2.13) lib/active_support/callbacks.rb:405:in `__run_callback' activesupport (3.2.13) lib/active_support/callbacks.rb:390:in `_run_call_callbacks' org/jruby/RubyBasicObject.java:1659:in `__send__' org/jruby/RubyKernel.java:2086:in `send' activesupport (3.2.13) lib/active_support/callbacks.rb:81:in `run_callbacks' actionpack (3.2.13) lib/action_dispatch/middleware/callbacks.rb:27:in `call' actionpack (3.2.13) lib/action_dispatch/middleware/remote_ip.rb:31:in `call' actionpack (3.2.13) lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call' actionpack (3.2.13) lib/action_dispatch/middleware/show_exceptions.rb:56:in `call' railties (3.2.13) lib/rails/rack/logger.rb:32:in `call_app' railties (3.2.13) lib/rails/rack/logger.rb:16:in `call' activesupport (3.2.13) lib/active_support/tagged_logging.rb:22:in `tagged' railties (3.2.13) lib/rails/rack/logger.rb:16:in `call' actionpack (3.2.13) lib/action_dispatch/middleware/request_id.rb:22:in `call' rack (1.4.5) lib/rack/methodoverride.rb:21:in `call' rack (1.4.5) lib/rack/runtime.rb:17:in `call' rack (1.4.5) lib/rack/lock.rb:15:in `call' rack-cache (1.2) lib/rack/cache/context.rb:136:in `forward' rack-cache (1.2) lib/rack/cache/context.rb:143:in `pass' rack-cache (1.2) lib/rack/cache/context.rb:155:in `invalidate' rack-cache (1.2) lib/rack/cache/context.rb:71:in `call!' rack-cache (1.2) lib/rack/cache/context.rb:51:in `call' railties (3.2.13) lib/rails/engine.rb:479:in `call' railties (3.2.13) lib/rails/application.rb:223:in `call' org/jruby/RubyBasicObject.java:1665:in `__send__' org/jruby/RubyKernel.java:2090:in `send' railties (3.2.13) lib/rails/railtie/configurable.rb:30:in `method_missing'
on 2013-09-06 20:30

on 2013-09-06 21:00

Hi Tim, Looks like that string is ISO-8859-1 encoded, not UTF-8. You might want to try https://github.com/whitequark/rack-utf8_sanitizer rather than dealing with it in your app. Rohit
on 2013-09-06 21:13

Great, thanks for the nudge, Rohit. I'll give it a shot and report back in case anyone else runs into this. Thanks for the quick response! Tim
on 2013-09-07 02:04

Maybe that can be related...or can help... https://groups.google.com/forum/#!topic/jruby-user... Regards El 06/09/2013 20:30, "Tim Griffin" <tim.griffin@enwood.ca> escribi:
on 2013-09-07 17:15

Hi Manuel, I don't think this issue is related to the one in the link you posted. $ irb jruby-1.7.4 :001 > s = "K Labb\xE9" => "K Labb\xE9" jruby-1.7.4 :002 > s.encode("UTF-8", "ISO-8859-1") => "K Labb" Rohit
on 2013-09-07 20:58

Hi Rigth... Its different El 07/09/2013 17:15, "Rohit Namjoshi" <rohit.namjoshi@gmail.com> escribi:
on 2013-09-08 00:23

Hi Rohit; From your test statement, it looks like I might not even have to go as far as integrating the rack-utf8-sanitizer (provided I know that the input is always arriving in ISO-8859-1), right? Your s.encode statement appears to do the trick. Of course, when I can't guarantee the encoding, rack-utf8-sanitizer is the better solution. I am a little surprised that with Rails support for UTF-8 and internationalization that there isn't a more built-in check/conversion against non-UTF-8 strings. I really didn't think that I'd encounter this problem and have to look for a solution *outside* of Rails. Tim