Gateway Upgraded

Ruby
21

On Nov 3, 2007, at 11:10 AM, Douglas Wells wrote:

This transition appears to be a success.

James Edward G. II

Well, there appears to be at least one little glitch remaining.
The following header line in the immediately referenced article
shouldn’t appear with an unencoded article body:

Content-Transfer-Encoding: Base64

I’ve deployed some fixes to the gateway this morning that I hope will
correct this issue. Please let me know if you see more problems.

James Edward G. II

22

I assume that’s because TMail is undoing the Base64 encoding for me
and not putting it back to match the transfer_encoding() of
outgoing. Do I have that right?

TMail does not handle the email body. In fact, it basically ignores
it beyond breaking it down by parts when you explicitly request it.

So this is what happens:

mail = TMail::Mail.load(“my_base64_encoded_body_email”)
body = mail.body # this will actually decode the body for you to be nice
mail.body = body # change the body text to an unencoded string
mail[‘Content-Transfer-Encoding’] == Base64 # => true

Basically, TMail has no way of knowning that the body text has changed
to a non Base64 string, it needs to be told that.

I think, in essence, this is what is happening on the gateway,
please correct me if I am wrong.

An (easier?) way, that preserves the body text and the transfer
encoding type would be to do:

outgoing_mail = TMail.parse($stdin.read)
outgoing_mail[‘x-header’] = “whatever you need to add”

Because when you do the outgoing.body = incomming.body TMail is being
nice and decoding it for you.

If you want the undecoded text and still do the assignment, you can do:

outgoingmail.body = incommingmail.quoted_body

I think I might make an alias for “undecoded_body” to “quoted_body” but
anyway.

Is there any reason you are making two TMail objects instead of just
making one new one and parsing in the incomming string and then adding
what you need?

What’s the best way for us to fix this? Should I not copy the
transfer encoding of the original message? Will TMail normalize all
of them for me? What does it normalize to? Maybe I should declare
that.

Just make one object and pull in the data and leave the body alone, I
think, would be the easiest.

Mikel, why is TMail not wrapping this? I know you added some checks
to ignore X-* headers, but shouldn’t it only ignore them if they
aren’t easily wrapped? This one has a lot of whitespace in it, so
it’s wrappable, right?

Hmm… I originally put in stuff to ignore the X-Headers completely,
this smelt a bit bad so then I changed this to wrapping anything that
has over 78 characters total that has a whitespace which is more RFC
compliant.

So, now it doesn’t ignore any headers… it tries to wrap them if they
have whitespace that it can wrap (ie, whitespace in the first 78
characters). If it doesn’t have whitespace within the first 78 chars
(including the header) it doesn’t wrap…

So, basically, this should be wrapping correctly.

But I am pretty sure you are using this version of trunk because that
is the same revision that fixed the reply-to problem you were having.

Could you send me an unencoded, unwrapped copy of the string you
insert so I can make a test case with it?

Regards

Mikel

23

On 11/2/07, James Edward G. II [email protected] wrote:

According to my scan of the log this morning, every email the gateway
sent since yesterday’s upgrade was accepted by Usenet. I was even
able to find messages modified by the gateway and they looked great
on comp.lang.ruby.

This transition appears to be a success.

Brilliant work. Out of curiosity, is there any part of the gateway
that’d be worth extracting as a more general library or utility?

martin

24

On Nov 5, 2007, at 6:45 PM, Martin DeMello wrote:

that’d be worth extracting as a more general library or utility?
Well, the gateway is now just a bit of glue code between TMail and
Net::NNTP, so I guess you could say that it has.

James Edward G. II

25

On Nov 5, 2007, at 7:00 PM, Mikel L. wrote:

What do you think James?
This might be a possibility.

Let’s give the new gateway a little more time in the oven before we
decide anything though.

People are still finding issues with the new code. When that
stabilizes, I want to rewrite the other half of the gateway using
what I have learned.

I’ll share my code after that and we can all talk about whether there
is anything worth salvaging.

James Edward G. II

26

Maybe we could make it an extension to TMail…

email.to_nntp

Or something?

Haven’t looked into this at all and right now for me the priority is
trying to make the documentation for TMail and fix exisitng bugs, but
I think that would be a useful feature to add.

What do you think James?

Regards

Mikel

27

On Nov 5, 2007, at 7:49 AM, James Edward G. II wrote:

able to find messages modified by the gateway and they looked

Content-Transfer-Encoding: Base64

I’ve deployed some fixes to the gateway this morning that I hope
will correct this issue. Please let me know if you see more problems.

I have deployed another set of fixes to the gateway tonight.

I’ve attempted to correct a reported problem where the name was being
dropped from “From” fields.

I’ve also made the code that adds the this-message-was-modified note
more careful. I now only add it to 7bit, 8bit, and binary encoded
bodies, so I don’t damage something like Base64 encoded content. The
note will now always appear in an X-Gateway-Modified header
regardless of body encoding, so we will have a means of tracking
this. I realize that means it’s not immediately visible for some
message types, but it seems safer to me than reencoding the message.

Please let me know if you locate further issues.

James Edward G. II