# Rake Environment variables gone missing

**URL:** <https://www.ruby-forum.com/t/rake-environment-variables-gone-missing/179673>\
**Category:** Ruby\
**Created:** [December 1, 2009, 5:16pm UTC](https://www.ruby-forum.com/t/rake-environment-variables-gone-missing/179673 "2009-12-01T17:16:51Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![hermans](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/f07891/32.png) [@hermans](https://www.ruby-forum.com/u/hermans)\
**Post date:** [December 1, 2009, 5:16pm UTC](https://www.ruby-forum.com/t/rake-environment-variables-gone-missing/179673/1 "2009-12-01T17:16:51Z")

</div>

I posted this about six months ago but the form seems to have gone  
missing.

Unfortunately, I have been sent back to work on the project from hell  
and am running into the same issue.

The previous posts didn’t produce much help as no one seems to be able  
to produce the problem but me.

I am running on windows-xp machine with ruby 1.8.6 (2008-08-11  
patchlevel 287) [i386-mswin32] and rake, version 0.8.1.

Here’s the issue. I read in a windows batch file that has a bunch of  
environment variable in it that are used in a hornets nest of Devstudio  
C++ projects. I want to parse the variables out and pass them to ENV so  
I can invoke incredibuild from rake.

Here’s the problem code.

File.foreach env\_batch\_file do |line|  
if /^\s_SET\s_([^\s=]+)\s\*=\s\*(.\*)$/i =~ line  
env\_count = ENV.size()  
key = $1  
value = resolve\_variables($2)

```
  key.strip!
  value.strip!
  already_has_key = ENV.has_key?(key)

  ENV[key] = value.dup
  env_count = env_count + 1

  if( env_count != ENV.size())
     if ( already_has_key )
          verbose("Variable already set.", 5)
       else
          verbose("ENV Hash whacked." , 1)
     end
  end

```

end

In this particular manifestation of the problem it happens when the ENV  
size grows to 360. Here’s some output:

360:  
In Value = %IIE\_SOURCE%\cen\_xml\_support\build\windows\static\release  
Rep = F:\source\iie\trunk  
Val = %IIE\_SOURCE%\cen\_xml\_support\build\windows\static\release  
Val = F:\source\iie\trunk\cen\_xml\_support\build\windows\static\release  
No quotes.  
Return -\>  
F:\source\iie\trunk\cen\_xml\_support\build\windows\static\release  
360:ENV[CEN\_XML\_SUPPORT\_REL\_LIB\_LOC] =  
F:\source\iie\trunk\cen\_xml\_support\build\windows\static\release  
ENV Hash whacked. 360:361 -\> CEN\_XML\_SUPPORT\_REL\_LIB\_LOC  
360

I should note that in the very next attempt the size of ENV goes to 27!  
So something is going seriously wrong here.

Any thoughts as to what specifically is failing.

---

<div class="post-metadata">

**Author:** ![hermans](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/f07891/32.png) [@hermans](https://www.ruby-forum.com/u/hermans)\
**Post date:** [December 2, 2009, 5:15pm UTC](https://www.ruby-forum.com/t/rake-environment-variables-gone-missing/179673/2 "2009-12-02T17:15:45Z")

</div>

On Dec 1, 11:16 am, Herman S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> patchlevel 287) [i386-mswin32] and rake, version 0.8.1.  
> env\_count = ENV.size()  
> if( env\_count != ENV.size())
> 
> ENV Hash whacked. 360:361 → CEN\_XML\_SUPPORT\_REL\_LIB\_LOC  
> 360
> 
> I should note that in the very next attempt the size of ENV goes to 27!  
> So something is going seriously wrong here.
> 
> ## Any thoughts as to what specifically is failing.
> 
> Posted viahttp://www.ruby-forum.com/.

Herman,

I found one reference ([http://blogs.msdn.com/oldnewthing/archive/](http://blogs.msdn.com/oldnewthing/archive/)  
2006/07/06/657868.aspx ) indicating that the Windows environment is  
limited to 64K, or 32Kcharacters in size. I’ll bet you hit that limit.

It probably broke for you where it worked for others because by  
resolving variables, you’re increasing the size of the values.

-T3ch.dude

---

<div class="post-metadata">

**Author:** ![hermans](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/f07891/32.png) [@hermans](https://www.ruby-forum.com/u/hermans)\
**Post date:** [December 2, 2009, 6:30pm UTC](https://www.ruby-forum.com/t/rake-environment-variables-gone-missing/179673/3 "2009-12-02T18:30:41Z")

</div>

t3ch.dude wrote:

> On Dec 1, 11:16�am, Herman S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> > patchlevel 287) [i386-mswin32] and rake, version 0.8.1.  
> > � � � env\_count = ENV.size()  
> > � � � if( env\_count != ENV.size())
> > 
> > ENV Hash whacked. 360:361 → CEN\_XML\_SUPPORT\_REL\_LIB\_LOC  
> > 360
> > 
> > I should note that in the very next attempt the size of ENV goes to 27!  
> > So something is going seriously wrong here.
> > 
> > ## Any thoughts as to what specifically is failing.
> > 
> > Posted viahttp://www.ruby-forum.com/.
> 
> Herman,
> 
> I found one reference ([http://blogs.msdn.com/oldnewthing/archive/](http://blogs.msdn.com/oldnewthing/archive/)  
> 2006/07/06/657868.aspx ) indicating that the Windows environment is  
> limited to 64K, or 32Kcharacters in size. I’ll bet you hit that limit.
> 
> It probably broke for you where it worked for others because by  
> resolving variables, you’re increasing the size of the values.
> 
> -T3ch.dude  
> Interesting. Doesn’t give me much hope though. I suppose I could try  
> it without resolving all variables. That means I’ll have to split the  
> environment between automation variables and devstudio variables. For  
> example to shared library paths are used by both the automation system  
> and devstudio. I’ll need those resolved at some point for the  
> automation system.

hmmm. Don’t think I’ll get to it today. But I will try soon.

Thanks.

---

<div class="post-metadata">

**Author:** ![hermans](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/f07891/32.png) [@hermans](https://www.ruby-forum.com/u/hermans)\
**Post date:** [December 2, 2009, 10:44pm UTC](https://www.ruby-forum.com/t/rake-environment-variables-gone-missing/179673/4 "2009-12-02T22:44:55Z")

</div>

Aldric G. wrote:

> Herman S. wrote:
> 
> > Here’s the issue. I read in a windows batch file that has a bunch of  
> > environment variable in it that are used in a hornets nest of Devstudio  
> > C++ projects. I want to parse the variables out and pass them to ENV so  
> > I can invoke incredibuild from rake.
> 
> Herman,  
> Could you convince me that you’re not doing any kind of useless steps,  
> here? How about just doing the parsing from within Ruby/Rake ?

Well the problem is that the vcproj files are all based on environment  
variables. The defines, the compiler options, the include paths, and  
the paths to the libraries for linking.

So you have things like  
…  
\<Tool  
Name=“VCCLCompilerTool”  
AdditionalOptions="$(GQT\_DRIVER\_DEB\_DEFINES)"  
…  
in the environment batch file it looks like this

SET GQT\_REL\_DEFINES=%COMMON\_DEFINES% %COMMON\_REL\_DEFINES% /D  
XERCES\_TMPLSINC %COMMON\_LIBRARY\_DEFINES%

The project was setup like this by the team in such a way that you have  
to run the batch script before you open the vcproj file. If you just  
double click it from an explorer window, for example, it won’t work.

Likewise if you’re automating the build, you need to have the  
environment resolved before you build. So if you want to shell out from  
rake to invoke the build you need to have that environment set.

In the passed, I used to chain the calling the environment script to the  
call to incredibuild.

Something like

result = system( “setEnvironment.bat;{incredibuild statement}” )

There were issues with that as I remember, so I wrote some ruby code to  
parse the environment batch file and set the environment. It works fine  
on smaller projects but this one is gianormous.

Finally, once I had all the environment variables, I could insure that  
the build verification tests and the publishing scripts were all pulling  
the same files as the build system had used. I like that; single point  
of failure and all.

So if I am going to try to slide under the limit ( if that indeed is my  
problem ) I’ll need to identify those variables which devstudio needs to  
have resolved vs. what variables I can read from rake at run time.

thanks again for the help. I did try today, but I wasn’t able to get  
under the limit. I’ll need a sharper knife.

---

<div class="post-metadata">

**Author:** ![hermans](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/f07891/32.png) [@hermans](https://www.ruby-forum.com/u/hermans)\
**Post date:** [December 2, 2009, 6:44pm UTC](https://www.ruby-forum.com/t/rake-environment-variables-gone-missing/179673/5 "2009-12-02T18:44:59Z")

</div>

Herman S. wrote:

> Here’s the issue. I read in a windows batch file that has a bunch of  
> environment variable in it that are used in a hornets nest of Devstudio  
> C++ projects. I want to parse the variables out and pass them to ENV so  
> I can invoke incredibuild from rake.

Herman,  
Could you convince me that you’re not doing any kind of useless steps,  
here? How about just doing the parsing from within Ruby/Rake ?

---

<div class="post-metadata">

**Author:** ![hermans](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/f07891/32.png) [@hermans](https://www.ruby-forum.com/u/hermans)\
**Post date:** [December 2, 2009, 11:15pm UTC](https://www.ruby-forum.com/t/rake-environment-variables-gone-missing/179673/6 "2009-12-02T23:15:43Z")

</div>

On Dec 2, 4:44 pm, Herman S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> So if I am going to try to slide under the limit ( if that indeed is my  
> problem ) I’ll need to identify those variables which devstudio needs to  
> have resolved vs. what variables I can read from rake at run time.
> 
> thanks again for the help. I did try today, but I wasn’t able to get  
> under the limit. I’ll need a sharper knife.

You know, since you’re testing the size of the ENV hash, you can test  
the sum of the sizes of the values and report it out, just to see if  
that is in fact the problem. I’d hate to have sent you down the wrong  
path… always best to test!

-T3ch.dude

---

<div class="post-metadata">

**Author:** ![hermans](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/f07891/32.png) [@hermans](https://www.ruby-forum.com/u/hermans)\
**Post date:** [December 4, 2009, 5:07pm UTC](https://www.ruby-forum.com/t/rake-environment-variables-gone-missing/179673/7 "2009-12-04T17:07:13Z")

</div>

t3ch.dude wrote:

> On Dec 2, 4:44�pm, Herman S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> > So if I am going to try to slide under the limit ( if that indeed is my  
> > problem ) I’ll need to identify those variables which devstudio needs to  
> > have resolved vs. what variables I can read from rake at run time.
> > 
> > thanks again for the help. �I did try today, but I wasn’t able to get  
> > under the limit. �I’ll need a sharper knife.
> 
> You know, since you’re testing the size of the ENV hash, you can test  
> the sum of the sizes of the values and report it out, just to see if  
> that is in fact the problem. I’d hate to have sent you down the wrong  
> path… always best to test!
> 
> -T3ch.dude

No question it’s the size.

I wrote a little ruby script to fill up the environment and it the env  
gets whacked as the size approaches 32k. It does seem like there is a  
little wriggle room depending on how big the string that I am adding  
over and over again is, it fails at different sizes.

require ‘utilities.rb’

i = 0  
s = “X”

100.times { s += “X”}

#puts s

def env\_size()  
result = 0  
ENV.each do | key, value |  
result += value.length();  
end  
return result  
end

j = 0  
whacked = false  
while ( ! whacked && j \< 100000)

env\_count = ENV.size()  
verbose( ENV.size().to\_s() + “:”, 5)  
key = j.to\_s()  
value = s;

already\_has\_key = ENV.has\_key?(key)  
ENV[key] = value.dup

verbose(ENV.size().to\_s()+“:ENV[” + key + "] = " + value, 11)  
env\_count = env\_count + 1

if( env\_count != ENV.size())  
if ( already\_has\_key )  
verbose(“Variable already set.”, 5)  
else  
verbose("ENV Hash whacked. " + ENV.size().to\_s() + “:” +  
env\_count.to\_s() + " → " + key, 1)  
whacked = true  
end  
end  
verbose( ENV.size().to\_s(), 5)  
verbose( "Nmbr of Env entries = " + ENV.size().to\_s(), 1)  
verbose( "Size of Environment = " + env\_size().to\_s(), 1)

j += 1  
end

Well ok, thanks for the help. I guess I have to find a work around.