# Ruby imap sync script error with attachments

**URL:** <https://www.ruby-forum.com/t/ruby-imap-sync-script-error-with-attachments/133966>\
**Category:** Ruby\
**Created:** [April 5, 2008, 7:02pm UTC](https://www.ruby-forum.com/t/ruby-imap-sync-script-error-with-attachments/133966 "2008-04-05T19:02:12Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [April 5, 2008, 7:02pm UTC](https://www.ruby-forum.com/t/ruby-imap-sync-script-error-with-attachments/133966/1 "2008-04-05T19:02:12Z")

</div>

I have found a ruby imap script online that i’m having some trouble  
with, its originally from  
[http://blog.wherenow.org/blog/2008/01/10/transferring-email-via-imap/](http://blog.wherenow.org/blog/2008/01/10/transferring-email-via-imap/)  
but I was hoping to get some help from the group with it.

I’m primarily a sysadmin who scripts in shell and some perl, i’m kinda  
lost in ruby.

Long story short i run a Scalix server and have moved mail hosts over  
to google apps. I have a bunch of imap accounts that I would like to  
sync to google apps (gmail for domains).

Unfortunately Scalix (the source IMAP server) adds the non-standard  
flag \X-Has-Attach as well as \X-Forwarded, contrary to RFC 3501.

I’ve seen in another forum where they add the following regex to  
imapsync to get around this

-regexflag ‘s/\X-\*//g’

But, I have about 250 mailboxes to migrate, imapsync and needing to  
specify all folders is out of the question.

Is there any way that you can help me out? Follows is the script and  
the error I get when it runs;

======  
#!/usr/bin/env ruby  
require ‘net/imap’

# Source server connection info.

SOURCE\_HOST = ‘192.168.1.150’  
SOURCE\_PORT = 143  
SOURCE\_SSL = false  
SOURCE\_USER = ‘removed\_email\_address@domain.invalid’  
SOURCE\_PASS = ‘password’

# Destination server connection info.

DEST\_HOST = ‘[imap.gmail.com](http://imap.gmail.com)’  
DEST\_PORT = 993  
DEST\_SSL = true  
DEST\_USER = ‘removed\_email\_address@domain.invalid’  
DEST\_PASS = ‘password’

# Mapping of source folders to destination folders. The key is the

name of the

# folder on the source server, the value is the name on the

destination server.

# Any folder not specified here will be ignored. If a destination

folder does

# not exist, it will be created.

SFOLDERS = {}  
DFOLDERS = {}

# Utility methods.

def dd(message)  
puts “[DEST: #{DEST\_HOST}] #{message}”  
end

def ds(message)  
puts “[SOURCE: #{SOURCE\_HOST}] #{message}”  
end

# Initialization

puts “[BACKUP] from ‘#{SOURCE\_USER}’ to ‘#{DEST\_USER}’”

# Connect and log into both servers.

ds ‘connecting…’  
source = Net::IMAP.new(SOURCE\_HOST, SOURCE\_PORT, SOURCE\_SSL)

ds ‘logging in…’  
source.login(SOURCE\_USER, SOURCE\_PASS)

dd ‘connecting…’  
dest = Net::IMAP.new(DEST\_HOST, DEST\_PORT, DEST\_SSL)

dd ‘logging in…’  
dest.login(DEST\_USER, DEST\_PASS)

ds ‘enumerating folders…’  
sfolderslist = source.list(‘_', '_’)  
dfolderslist = dest.list(‘_', '_’)

ds ‘populating folders lists…’

# List all source folders

i=0  
while sfolderslist[i]!=nil  
SFOLDERS[sfolderslist[i].name] = sfolderslist[i].name  
i=i+1  
end

# List all destination folders no more present on the source

i=0  
while dfolderslist[i]!=nil  
if SFOLDERS[dfolderslist[i].name] != dfolderslist[i].name  
DFOLDERS[dfolderslist[i].name] = dfolderslist[i].name  
end  
i=i+1  
end

# Loop through destination folders and remove unused ones

#DFOLDERS.each do |source\_folder, dest\_folder|

# Open destination folder in read-write mode

#begin  
#dd “removing folder ‘#{dest\_folder}’”  
#dest.select(dest\_folder)  
#dest.delete(dest\_folder)  
#end  
#end

# Loop through source folders and copy messages.

SFOLDERS.each do |source\_folder, dest\_folder|

# Open source folder in read-only mode.

begin  
ds “selecting folder ‘#{source\_folder}’…”  
source.examine(source\_folder)  
rescue =\> e  
ds “error: select failed: #{e}”  
next  
end

# Open (or create) destination folder in read-write mode.

begin  
dd “selecting folder ‘#{dest\_folder}’…”  
dest.select(dest\_folder)  
rescue =\> e  
begin  
dd “folder not found; creating…”  
dest.create(dest\_folder)  
dest.select(dest\_folder)  
rescue =\> ee  
dd “error: could not create folder: #{e}”  
next  
end  
end

# Build a lookup hash of all message ids present in source and in the

destination folder.  
source\_info = {}  
dest\_info = {}

dd ‘analyzing destination existing messages…’  
uids = dest.uid\_search([‘ALL’])  
if uids.length \> 0  
dest.uid\_fetch(uids, [‘ENVELOPE’]).each do |data|  
dest\_info[data.attr[‘ENVELOPE’].message\_id] = true  
end  
end

ds ‘analyzing source existing messages…’  
uidd = source.uid\_search([‘ALL’])  
if uidd.length \> 0  
source.uid\_fetch(uidd, [‘ENVELOPE’]).each do |data|  
source\_info[data.attr[‘ENVELOPE’].message\_id] = true  
end  
end

# Loop through all messages in the destination folder.

uida = dest.uid\_search([‘ALL’])  
if uida.length \> 0  
dest.uid\_fetch(uida, [‘ENVELOPE’]).each do |data|

mid2 = data.attr[‘ENVELOPE’].message\_id

# If this message is included in both folders, skip it.

next if source\_info[mid2] && dest\_info[mid2]

# Else delete the message from the destination folder.

dd “removing message #{mid2}…”  
dest.uid\_store(data.attr[‘UID’], ‘+FLAGS’, [:Deleted])  
dest.expunge  
end

end

# Loop through all messages in the source folder.

uids = source.uid\_search([‘ALL’])  
if uids.length \> 0  
source.uid\_fetch(uids, [‘ENVELOPE’]).each do |data|  
mid = data.attr[‘ENVELOPE’].message\_id

# If this message is already in the destination folder, skip it.

next if dest\_info[mid]

# Download the full message body from the source folder.

ds “downloading message #{mid}…”  
msg = source.uid\_fetch(data.attr[‘UID’], [‘RFC822’, ‘FLAGS’,  
‘INTERNALDATE’]).first

# Append the message to the destination folder, preserving flags and

# internal timestamp.

dd “storing message #{mid}…”  
dest.append(dest\_folder, msg.attr[‘RFC822’], msg.attr[‘FLAGS’],  
msg.attr[‘INTERNALDATE’])  
end  
end

source.close  
dest.close  
end

# puts ‘[BACKUP] done’

error

=====================

[root@acserver1 ruby]# ./is2.rb  
[BACKUP] from ‘removed\_email\_address@domain.invalid’ to ‘removed\_email\_address@domain.invalid’  
[SOURCE: 192.168.1.150] connecting…  
[SOURCE: 192.168.1.150] logging in…  
[DEST: [imap.gmail.com](http://imap.gmail.com)] connecting…  
[DEST: [imap.gmail.com](http://imap.gmail.com)] logging in…  
[SOURCE: 192.168.1.150] enumerating folders…  
[SOURCE: 192.168.1.150] populating folders lists…  
[SOURCE: 192.168.1.150] selecting folder ‘Drafts’…  
[DEST: [imap.gmail.com](http://imap.gmail.com)] selecting folder ‘Drafts’…  
[DEST: [imap.gmail.com](http://imap.gmail.com)] analyzing destination existing messages…  
[SOURCE: 192.168.1.150] analyzing source existing messages…  
[SOURCE: 192.168.1.150] downloading message  
[LDC8AD85A83E84fd1B6A0B6803EB405C4.1196803592.server1.domain.com@MHS](mailto:LDC8AD85A83E84fd1B6A0B6803EB405C4.1196803592.server1.domain.com@MHS)…  
[DEST: [imap.gmail.com](http://imap.gmail.com)] storing message  
[LDC8AD85A83E84fd1B6A0B6803EB405C4.1196803592.server1.domain.com@MHS](mailto:LDC8AD85A83E84fd1B6A0B6803EB405C4.1196803592.server1.domain.com@MHS)…  
/usr/lib/ruby/1.8/net/imap.rb:952:in `get_tagged_response': Invalid Arguments: Unable to parse flag \X-has-attach (Net::IMAP::BadResponseError) from /usr/lib/ruby/1.8/net/imap.rb:998:in `send\_command’  
from /usr/lib/ruby/1.8/net/imap.rb:986:in `synchronize' from /usr/lib/ruby/1.8/net/imap.rb:1004:in `send\_command’  
from /usr/lib/ruby/1.8/net/imap.rb:598:in `append' from ./is2.rb:162 from ./is2.rb:148:in `each’  
from ./is2.rb:148  
from ./is2.rb:82:in `each’  
from ./is2.rb:82

==============================================================================

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [April 5, 2008, 10:23pm UTC](https://www.ruby-forum.com/t/ruby-imap-sync-script-error-with-attachments/133966/2 "2008-04-05T22:23:05Z")

</div>

On Apr 5, 2008, at 10:00 , removed\_email\_address@domain.invalid wrote:

> I’ve seen in another forum where they add the following regex to  
> imapsync to get around this
> 
> -regexflag ‘s/\X-\*//g’
> 
> But, I have about 250 mailboxes to migrate, imapsync and needing to  
> specify all folders is out of the question.

I’d still use imapsync. It is the only thing out there I’ve found that  
does a good job, esp considering it is a real sync (you can rerun to  
sync anything new). Use ruby (or whatever) to generate a script that  
invokes all the imapsync’s you need. It should be a fairly simple  
matter to iterate through /etc/passwd’s users and generate an imapsync  
for all of them (not for each and every folder). Here is what I used a  
while back:

#!/bin/bash

user1=$1; shift  
user2=$1; shift

imapsync --host1 localhost --user1 $user1 --host2 xxx --user2 $user2 –  
ssl1 --ssl2

* * *

I didn’t have 250 users, otherwise I’d have written what I suggest  
above and given it to you. 🙂

---

<div class="post-metadata">

**Author:** ![unknown](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/4bbf92/32.png) [@unknown](https://www.ruby-forum.com/u/unknown)\
**Post date:** [April 6, 2008, 8:20pm UTC](https://www.ruby-forum.com/t/ruby-imap-sync-script-error-with-attachments/133966/3 "2008-04-06T20:20:58Z")

</div>

On Apr 5, 4:22 pm, Ryan D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> On Apr 5, 2008, at 10:00 , removed\_email\_address@domain.invalid wrote:

> I’d still use imapsync. It is the only thing out there I’ve found that  
> does a good job, esp considering it is a real sync (you can rerun to  
> sync anything new). Use ruby (or whatever) to generate a script that  
> invokes all the imapsync’s you need. It should be a fairly simple  
> matter to iterate through /etc/passwd’s users and generate an imapsync  
> for all of them (not for each and every folder). Here is what I used a  
> while back:

Fair enough, thought I could get away with an easy loop ;-), i’ll look  
into imapsync some more.

Thanks for your reply.