# Openssl and SHA\*

**URL:** <https://www.ruby-forum.com/t/openssl-and-sha/162008>\
**Category:** Ruby\
**Created:** [March 4, 2009, 8:07pm UTC](https://www.ruby-forum.com/t/openssl-and-sha/162008 "2009-03-04T20:07:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![chobolt](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/48db29/32.png) [@chobolt](https://www.ruby-forum.com/u/chobolt)\
**Post date:** [March 4, 2009, 8:07pm UTC](https://www.ruby-forum.com/t/openssl-and-sha/162008/1 "2009-03-04T20:07:29Z")

</div>

Hi,

## I’m quite new to ruby, and I’m facing a problem I can’t seem to be able to solve by myself… I’m comparing openssl sha1 hash results from a linux command line, to ruby ones :

cmd line :  
openssl dgst -sha1 my\_file

## ruby : require ‘digest/sha1’ puts Digest::SHA1.hexdigest(File.read(“my\_file”))

I increase the file and run it again, and again.  
The hashes are similars until the file size reaches 512Mo, from then  
they differs.  
I tried several sha versions (sha256…sha512) and the problem is the  
same.  
However with MD5, I have no problem.

Anyone has an idea if I’m doing something wrong here ?  
Thanks a lot !

ChoBolT

---

<div class="post-metadata">

**Author:** ![chobolt](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/48db29/32.png) [@chobolt](https://www.ruby-forum.com/u/chobolt)\
**Post date:** [March 4, 2009, 10:13pm UTC](https://www.ruby-forum.com/t/openssl-and-sha/162008/2 "2009-03-04T22:13:59Z")

</div>

Philippe Chotard wrote:

> ## I’m comparing openssl sha1 hash results from a linux command line, to ruby ones :
> 
> cmd line :  
> openssl dgst -sha1 my\_file
> 
> ## ruby : require ‘digest/sha1’ puts Digest::SHA1.hexdigest(File.read(“my\_file”))
> 
> I increase the file and run it again, and again.  
> The hashes are similars until the file size reaches 512Mo, from then  
> they differs.

Strange. First, try doing it in two stages:

str = File.read(“my\_file”)  
puts str.size  
puts Digest::SHA1.hexdigest(str)

This may give you a clue if File.read is misbehaving. However this is  
unlikely if Digest::MD5 is fine.

But in any case, reading 512MB of data into RAM just to calculate SHA1  
is very wasteful. I suggest you recode it:

puts Digest::SHA1.file(“my\_file”).hexdigest

or read the file in blocks:

d = Digest::SHA1.new  
File.open(“my\_file”) do |f|  
while chunk = f.read(65536)  
d \<\< chunk  
end  
end  
puts d.hexdigest

If you _still_ get the same answer, then perhaps the command-line tool  
you are comparing against is at fault! Most Linux systems have at least  
two:

sha1sum   
openssl sha1

so you can see if those agree or disagree, too.

On my box (Ubuntu Hardy, ruby-1.8.6p114 compiled from source):

$ ls -l ubuntu-8.04-desktop-i386.iso  
-rw-r–r-- 1 brian brian 733079552 Apr 24 2008  
ubuntu-8.04-desktop-i386.iso  
$ sha1sum ubuntu-8.04-desktop-i386.iso  
53a07a006d791f7fddc6d53879e826934f73bc0f ubuntu-8.04-desktop-i386.iso  
$ openssl dgst -sha1 ubuntu-8.04-desktop-i386.iso  
SHA1(ubuntu-8.04-desktop-i386.iso)=  
53a07a006d791f7fddc6d53879e826934f73bc0f  
$ irb  
irb(main):001:0\> require ‘digest/sha1’  
=\> true  
irb(main):002:0\>  
Digest::SHA1.file(“ubuntu-8.04-desktop-i386.iso”).hexdigest  
=\> “53a07a006d791f7fddc6d53879e826934f73bc0f”  
irb(main):003:0\> d = Digest::SHA1.new  
=\> #\<Digest::SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709\>  
irb(main):004:0\> File.open(“ubuntu-8.04-desktop-i386.iso”) do |f|  
irb(main):005:1\* while chunk = f.read(65536)  
irb(main):006:2\> d \<\< chunk  
irb(main):007:2\> end  
irb(main):008:1\> end  
=\> nil  
irb(main):009:0\> d.hexdigest  
=\> “53a07a006d791f7fddc6d53879e826934f73bc0f”  
irb(main):010:0\>

So I can’t see any problem. However I don’t really have enough RAM to  
read the file all in at once without swapping badly. It’s possible that  
Digest::SHA1 barfs when given a string \> 512MB.

Regards,

Brian.

---

<div class="post-metadata">

**Author:** ![chobolt](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/48db29/32.png) [@chobolt](https://www.ruby-forum.com/u/chobolt)\
**Post date:** [March 5, 2009, 1:23pm UTC](https://www.ruby-forum.com/t/openssl-and-sha/162008/3 "2009-03-05T13:23:57Z")

</div>

Brian C. wrote:

> But in any case, reading 512MB of data into RAM just to calculate SHA1  
> is very wasteful. I suggest you recode it:
> 
> puts Digest::SHA1.file(“my\_file”).hexdigest

Thanks for your response Brian.  
Indeed, using this method I got the right hash. So it looks like as you  
said, that the problem is appearing when sha is handling 512MB+ strings.

I’ll do some further testing on other systems and versions (Using ruby  
1.9.0 on a debian lenny)

Thanks !