# Problem writing inverse to java xor encoding function

**URL:** <https://www.ruby-forum.com/t/problem-writing-inverse-to-java-xor-encoding-function/130590>\
**Category:** Ruby\
**Created:** [February 29, 2008, 4:56pm UTC](https://www.ruby-forum.com/t/problem-writing-inverse-to-java-xor-encoding-function/130590 "2008-02-29T16:56:22Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![bensinc](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/8e7dd6/32.png) [@bensinc](https://www.ruby-forum.com/u/bensinc)\
**Post date:** [February 29, 2008, 4:56pm UTC](https://www.ruby-forum.com/t/problem-writing-inverse-to-java-xor-encoding-function/130590/1 "2008-02-29T16:56:22Z")

</div>

Hi,  
I’m trying to create an inverse to the following Java function in Ruby.

public static void encXORPass(byte[] raw, final int offset, final int  
size, int key)  
{  
int stop = size-8;  
int pos = 4 + offset;  
int edx;  
int ecx = key; // Initial xor key

```
while (pos < stop)
{

```

// get 4 bytes from raw into an int  
edx = (raw[pos] & 0xFF);  
edx |= (raw[pos+1] & 0xFF) \<\< 8;  
edx |= (raw[pos+2] & 0xFF) \<\< 16;  
edx |= (raw[pos+3] & 0xFF) \<\< 24;

// do things  
ecx += edx;  
edx ^= ecx;

// put the xor’d int back into raw  
raw[pos++] = (byte) (edx & 0xFF);  
raw[pos++] = (byte) (edx \>\> 8 & 0xFF);  
raw[pos++] = (byte) (edx \>\> 16 & 0xFF);  
raw[pos++] = (byte) (edx \>\> 24 & 0xFF);  
}  
// store the final key in the last 4 bytes of raw  
raw[pos++] = (byte) (ecx & 0xFF);  
raw[pos++] = (byte) (ecx \>\> 8 & 0xFF);  
raw[pos++] = (byte) (ecx \>\> 16 & 0xFF);  
raw[pos++] = (byte) (ecx \>\> 24 & 0xFF);  
}

I’m basically walking through the same loop backwards, but for some  
reason my code doesn’t currently completely decode the data. By  
“completely” I mean a few bytes into the loop it actually converges to  
the correct values, but not at the beginning. I’ve been trying to figure  
out why for weeks and I’m not making any progress so thought I’d post it  
here.

Here is an example of a correct decoded sequence, followed by my  
incorrect decoded sequence (yes they are different if you go far enough  
to the right):

00854130CE21C60000BD7F26062B09315C26E24D80378DD9FB568AF57C765D9A2C90A6B0124EA36014881A48A185B44FC9A6D922D03E1F91FB0468819721E639E21AC617D44D3A7E952C2211EDB36ABCFC81B51E8AC205DCC750D7EA0C18F49CE8A119A8DA67591C97B5B7D6C9EF61F7F25E6EBDC10EA0BAA1F388D3210198B1A66B1E09437E3AA2204E95DD29FC9CC37720B6AD97F7E0BD0731C3725F3B6566FEC6F2CD5473468F2700E8436B286DACD9FE6C8B9F9EADDF  
00854130CE21C60000BD7F26062B09315C26E24D80378DD9FB568AF57C765D9A2C90A6B0124EA36014881A48A185B44FC9A6D922D03E1F91FB0468819721E639E21AC617D44D3A7E952C2211EDB36ABCFC81B51E8AC205DCC750D7EA0C18F49CE8A119A8DA67591C97B5B7D6C9EF61F7F25E6EBDC10EA0BAA1F3883321019841A66B1EF1437E3ADE204E95C329FC1CCC772056AA97F71CBE07313C735FBB9A66FE06EDCD5493418F271FE8C3EBE86CEC41A6A4B69F9EADDF

The relevant ruby code is attached,  
Any insight appreciated,  
Cheers.

---

<div class="post-metadata">

**Author:** ![bensinc](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/8e7dd6/32.png) [@bensinc](https://www.ruby-forum.com/u/bensinc)\
**Post date:** [February 29, 2008, 5:56pm UTC](https://www.ruby-forum.com/t/problem-writing-inverse-to-java-xor-encoding-function/130590/2 "2008-02-29T17:56:04Z")

</div>

On Feb 29, 10:56 am, Ben S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> ```
> edx ^= ecx;
> raw[pos++] = (byte) (ecx >> 16 & 0xFF);
> 
> ```
> 
> Attachments:[http://www.ruby-forum.com/attachment/1502/xor.rb](http://www.ruby-forum.com/attachment/1502/xor.rb)
> 
> –  
> Posted viahttp://www.ruby-forum.com/.

FYI  
XOR(XOR(x)) == x

so to decode just run the encoded string through the xor

Cheers

---

<div class="post-metadata">

**Author:** ![bensinc](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/8e7dd6/32.png) [@bensinc](https://www.ruby-forum.com/u/bensinc)\
**Post date:** [February 29, 2008, 9:21pm UTC](https://www.ruby-forum.com/t/problem-writing-inverse-to-java-xor-encoding-function/130590/3 "2008-02-29T21:21:57Z")

</div>

Chris H. wrote:

> On Feb 29, 10:56 am, Ben S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> > ```
> > edx ^= ecx;
> > raw[pos++] = (byte) (ecx >> 16 & 0xFF);
> > 
> > ```
> > 
> > Attachments:[http://www.ruby-forum.com/attachment/1502/xor.rb](http://www.ruby-forum.com/attachment/1502/xor.rb)
> > 
> > –  
> > Posted viahttp://www.ruby-forum.com/.
> 
> FYI  
> XOR(XOR(x)) == x
> 
> so to decode just run the encoded string through the xor
> 
> Cheers

Thanks, yeah I know that, but that requires knowing the original ‘key’  
does it not? The problem is the Ruby program only knows the result after  
encoding which as far as I’m aware doesn’t include the original key that  
was used for encoding, only the modified key as it is after the pass.  
But I just had a thought, it’s possible the Java app only uses one set  
key, I’ll see if that’s the case and then I could just hard-code it in.

Cheers

---

<div class="post-metadata">

**Author:** ![bensinc](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/8e7dd6/32.png) [@bensinc](https://www.ruby-forum.com/u/bensinc)\
**Post date:** [February 29, 2008, 9:53pm UTC](https://www.ruby-forum.com/t/problem-writing-inverse-to-java-xor-encoding-function/130590/4 "2008-02-29T21:53:33Z")

</div>

Ben S. wrote:

> > But I just had a thought, it’s possible the Java app only uses one set  
> > key, I’ll see if that’s the case and then I could just hard-code it in.
> 
> Argh, the ‘key’ is different every time.

OK, I just needed to change where it started, ie. the two lines:

ecx = bytes2int(buf[size-4,4])  
pos=size-8

to:

ecx = bytes2int(buf[size-8,4])  
pos=size-12

I can’t believe, it works perfectly now.

---

<div class="post-metadata">

**Author:** ![bensinc](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/8e7dd6/32.png) [@bensinc](https://www.ruby-forum.com/u/bensinc)\
**Post date:** [February 29, 2008, 9:36pm UTC](https://www.ruby-forum.com/t/problem-writing-inverse-to-java-xor-encoding-function/130590/5 "2008-02-29T21:36:40Z")

</div>

> But I just had a thought, it’s possible the Java app only uses one set  
> key, I’ll see if that’s the case and then I could just hard-code it in.

Argh, the ‘key’ is different every time.