Forum: JRuby Time and microseconds

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Chuck R. (Guest)
on 2009-06-01 19:34
(Received via mailing list)
Charlie,

we chatted about this briefly on irc but I guess I don't understand
why the Time class doesn't generate microseconds. I've been calling
System.nanoTime and adding it to my calls to Time.now and it has been
pretty good. Why wouldn't a patch like this work?

--- RubyTime.java  2009-05-29 12:26:32.000000000 -0500
+++ RubyTime.java.new  2009-06-01 10:28:42.000000000 -0500
@@ -169,7 +169,7 @@
              DateTimeZone dtz = getLocalTimeZone(runtime);
              DateTime dt = new DateTime(dtz);
              RubyTime rt =  new RubyTime(runtime, klass, dt);
-            rt.setUSec(0);
+            rt.setUSec(((System.nanoTime() / 1000) % 1000000) /
1000000);

              return rt;
          }

If I recall, you mentioned that suspending/sleeping the VM caused
System.nanoTime to freak out. Am I remembering correctly?

cr


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Gary W. (Guest)
on 2009-06-01 19:50
(Received via mailing list)
Just saw this:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6480016

Specifically it states in the evaluation:

The specification of nanoTime is very clear that it
does not bear any relation to absolute time:

"This method can only be used to measure elapsed time and is not related
to any other notion of system or wall-clock time. The value returned
represents nanoseconds since some fixed but arbitrary time (perhaps in
the future, so values may be negative)."

See also
http://blogs.sun.com/dholmes/entry/inside_the_hots...
Posted Date : 2006-10-10 18:08:29.0

Hope that helps,
Gary


Chuck R. wrote:
>              DateTimeZone dtz = getLocalTimeZone(runtime);
> System.nanoTime to freak out. Am I remembering correctly?
>
> cr
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Charles Oliver N. (Guest)
on 2009-06-01 19:54
(Received via mailing list)
Chuck R. wrote:
>              DateTimeZone dtz = getLocalTimeZone(runtime);
>              DateTime dt = new DateTime(dtz);
>              RubyTime rt =  new RubyTime(runtime, klass, dt);
> -            rt.setUSec(0);
> +            rt.setUSec(((System.nanoTime() / 1000) % 1000000) / 1000000);
>
>              return rt;
>          }

This is actually setting it to whatever nanTime happens to be at any
given moment, which will be continuously increasing. It's not really a
valid nano/usec component of the current time unless you're tracking it
from a beginning point, which is where we run into problems.

> If I recall, you mentioned that suspending/sleeping the VM caused
> System.nanoTime to freak out. Am I remembering correctly?

That is correct; nanotime does not increment while a machine/VM is
sleeping, so you get peculiar results if it is the only thing used for
calculating time.

I know Tom would like to get sub-ms precision back into Time, so we're
open to finding a patch that does that without having suspend issues.

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Chuck R. (Guest)
on 2009-06-01 20:19
(Received via mailing list)
On Jun 1, 2009, at 10:54 AM, Charles Oliver N. wrote:

>>             DateTime dt = new DateTime(dtz);
> tracking it from a beginning point, which is where we run into
> we're open to finding a patch that does that without having suspend
> issues.

I see. Gary's response clued me in.

I'll see if I can't come up with a solution to get back microseconds.

cr


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Vladimir S. (Guest)
on 2009-06-01 20:31
(Received via mailing list)
Hi Chuck,

On Mon, Jun 1, 2009 at 5:33 PM, Chuck R. <removed_email_address@domain.invalid>
wrote:

> If I recall, you mentioned that suspending/sleeping the VM caused
> System.nanoTime to freak out. Am I remembering correctly?

I tried really hard to implement proper microseconds in JRuby's Time,
and for a while I thought that it worked, but at the end, way too much
magic and incorrect work in virtual environments *and* during the time
shifts (winter->summer time, etc). So now I'm convinced that it is
close to impossible to implement proper microseconds in pure-Java (via
nanoSeconds()). nanoSeconds specifically doesn't deal with *Time*,
only with time intervals, and it cannot be used to construct Time
objects.

The only practical solution would probably be to use JFFI to call
underlying platform API to obtain the time with needed precision. The
major drawback here is that such implementation would probably slower.

Thanks,
  --Vladimir

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Gary W. (Guest)
on 2009-06-01 20:47
(Received via mailing list)
Found a thread from 2006 stating some of the (what seem to be many)
issues with nanoTime():
http://www.velocityreviews.com/forums/t151271-p-sy...

like supposedly:
* System.nanoTime() uses QueryPerformanceCounters() on Windows and that
this function is known to have problems on Athlon64 multicore systems

Also here:
http://osdir.com/ml/java.jsr.166-concurrency/2003-...
which states:

"During my investigations I also found out
that on Windows platforms some chipsets cause leaps forward in time (up
to a
few seconds) when using the QueryPerformanceCounter-Call (which is -to
my
knowledge- the only way to measure in nanoseconds on Windows). This
behavior is
documented under:"

http://support.microsoft.com/default.aspx?scid=KB;...

and that says:

"Programs should watch for an unexpected jump by comparing the change in
time as determined by successive calls to QueryPerformanceCounter with
the change in time as determined by successive calls to the GetTickCount
function. If there is a significant jump that is based on
QueryPerformanceCounter(), but no similar increase that is based on
GetTickCount, then it can be assumed that the performance counter just
jumped forward. The code sample at the end of this article demonstrates
how to do this. This operating system's behavior is by design. The
performance counter adjustment is necessary when the operating system
obtains unreliable data from the chipset."

I just downloaded the latest Java 1.6 source
(openjdk-6-src-b16-24_apr_2009) and there are three occurrences of
QueryPerformanceCounters() in hotspot/src/os/windows/vm/os_windows.cpp
and one in jdk/src/windows/bin/java_md.c.

You got to love the comment "the best we can do", which appears to
describe the issue Microsoft described above:

hotspot/src/os/windows/vm/os_windows.cpp-#define NANOS_PER_SEC
CONST64(1000000000)
hotspot/src/os/windows/vm/os_windows.cpp-#define NANOS_PER_MILLISEC
1000000
hotspot/src/os/windows/vm/os_windows.cpp-jlong os::javaTimeNanos() {
hotspot/src/os/windows/vm/os_windows.cpp-  if (!has_performance_count) {
hotspot/src/os/windows/vm/os_windows.cpp-    return javaTimeMillis() *
NANOS_PER_MILLISEC; // the best we can do.
hotspot/src/os/windows/vm/os_windows.cpp-  } else {
hotspot/src/os/windows/vm/os_windows.cpp-    LARGE_INTEGER
current_count;
hotspot/src/os/windows/vm/os_windows.cpp:
QueryPerformanceCounter(&current_count);
hotspot/src/os/windows/vm/os_windows.cpp-    double current =
as_long(current_count);
hotspot/src/os/windows/vm/os_windows.cpp-    double freq =
performance_frequency;
hotspot/src/os/windows/vm/os_windows.cpp-    jlong time =
(jlong)((current/freq) * NANOS_PER_SEC);
hotspot/src/os/windows/vm/os_windows.cpp-    return time;
hotspot/src/os/windows/vm/os_windows.cpp-  }
hotspot/src/os/windows/vm/os_windows.cpp-}


In addition, here is the javadoc comment in the latest 1.6 source
(openjdk-6-src-b16-24_apr_2009) (I could have gotten from API docs
somewhere, but I was in there anyway) which notes an issue with
nanoTime() with spans of time greater than 292 years (2^63 nanoseconds):

    /**
     * Returns the current value of the most precise available system
     * timer, in nanoseconds.
     *
     * <p>This method can only be used to measure elapsed time and is
     * not related to any other notion of system or wall-clock time.
     * The value returned represents nanoseconds since some fixed but
     * arbitrary time (perhaps in the future, so values may be
     * negative).  This method provides nanosecond precision, but not
     * necessarily nanosecond accuracy. No guarantees are made about
     * how frequently values change. Differences in successive calls
     * that span greater than approximately 292 years (2<sup>63</sup>
     * nanoseconds) will not accurately compute elapsed time due to
     * numerical overflow.
     *
     * <p> For example, to measure how long some code takes to execute:
     * <pre>
     *   long startTime = System.nanoTime();
     *   // ... the code being measured ...
     *   long estimatedTime = System.nanoTime() - startTime;
     * </pre>
     *
     * @return The current value of the system timer, in nanoseconds.
     * @since 1.5
     */
    public static native long nanoTime();

Hope that helps!

Gary


Chuck R. wrote:
>>> +++ RubyTime.java.new    2009-06-01 10:28:42.000000000 -0500
>> This is actually setting it to whatever nanTime happens to be at any
>>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Wayne M. (Guest)
on 2009-06-01 20:51
(Received via mailing list)
2009/6/2 Vladimir S. <removed_email_address@domain.invalid>:
> The only practical solution would probably be to use JFFI to call
> underlying platform API to obtain the time with needed precision. The
> major drawback here is that such implementation would probably slower.

If you want to see how much slower, try running
bench/ffi/bench_gettimeofday.rb in 1.3.0RC2 or 1.3.0 on anything but
windows.

That is using the ruby FFI interface, but it'll be similar to the java
version.  On my machine the gettimeofday call is about 60% slower than
Time.now.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Chuck R. (Guest)
on 2009-06-01 22:48
(Received via mailing list)
On Jun 1, 2009, at 11:31 AM, Vladimir S. wrote:

> magic and incorrect work in virtual environments *and* during the time
> shifts (winter->summer time, etc). So now I'm convinced that it is
> close to impossible to implement proper microseconds in pure-Java (via
> nanoSeconds()). nanoSeconds specifically doesn't deal with *Time*,
> only with time intervals, and it cannot be used to construct Time
> objects.
>
> The only practical solution would probably be to use JFFI to call
> underlying platform API to obtain the time with needed precision. The
> major drawback here is that such implementation would probably slower.

I have an idea. Let me know if this sounds nuts.


1. When the RubyRuntime is instantiated, save the current
System.nanoTime as  a "nano epoch" and do the "expensive" jni call to
the underlying platform and save that as a "platform epoch". There
will be slippage here due to the call overhead but we'll assume the
call takes constant time.

# I assume get_time_of_day returns a 19 digit number to represent
nanoseconds just
# like nanoTime does
nano_epoch = System.nanoTime
platform_epoch = expensive_call_to_get_time_of_day

# round the platform time down to the nearest second
usec_remainder = (platform_epoch % 1000000000) / 1000

# sets nanoTime epoch to be closest to the nearest second
# relative to the platform's time
nano_epoch -= usec_remainder


2. Calls to Time.now will populate the +usec+ field by doing a simple
calculation.

def get_usec
   # nanoTime returns a 19 digit number where the last 3 digits are
always 0
   usec = ((System.nanoTime - nano_epoch) % 1000000000) / 1000
end

3. JRuby registers to receive notifications when the VM is awakened
from suspension (which I am assuming is possible). When it awakes, it
redoes the computation outlined in step 1.



Or am I crazy that this will work?

cr

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Gary W. (Guest)
on 2009-06-01 23:08
(Received via mailing list)
Chuck R. wrote:
>
> I have an idea. Let me know if this sounds nuts.
>
>
> 1. When the RubyRuntime is instantiated, save the current
> System.nanoTime as ...
>
>

and I think that is where it breaks down, because System.nanoTime is not
guaranteed to be accurate on all platforms. Even if everything after
that point was ok, it would continuously give an inaccurate number if
the number was based on nanoTime.

That's not from what I read- I'm not claiming to be an expert on it.
Read previous mails for more info as to why.

Hope that helps,

Gary

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Gary W. (Guest)
on 2009-06-01 23:09
(Received via mailing list)
Gary W. wrote:
> That's not from what I read

I meant "That from what I read". Gees- I need more sleep!

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Chuck R. (Guest)
on 2009-06-01 23:22
(Received via mailing list)
On Jun 1, 2009, at 2:07 PM, Gary W. wrote:

> and I think that is where it breaks down, because System.nanoTime is
> not guaranteed to be accurate on all platforms. Even if everything
> after that point was ok, it would continuously give an inaccurate
> number if the number was based on nanoTime.
>
> That's not from what I read- I'm not claiming to be an expert on it.
> Read previous mails for more info as to why.

I understand that. So we have three choices.

1. Detect the unreliable platforms and use the current scheme that is
accurate to the millisecond. Otherwise, use my mechanism or whatever
works.

2. Use the newer scheme to generate microseconds with a documented
caveat that they may be inaccurate on specific platforms.

#1 is probably the better choice. Use the greater precision where it
can be trusted with a fallback mode.

cr


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Vladimir S. (Guest)
on 2009-06-02 00:22
(Received via mailing list)
On Mon, Jun 1, 2009 at 8:46 PM, Chuck R. <removed_email_address@domain.invalid>
wrote:
>
>
> 2. Calls to Time.now will populate the +usec+ field by doing a simple
> calculation.
>
> def get_usec
>  # nanoTime returns a 19 digit number where the last 3 digits are always 0
>  usec = ((System.nanoTime - nano_epoch) % 1000000000) / 1000
> end

That's how it *was* implemented some time ago in JRuby, and it worked
most of the time, but was breaking during time shifts and VM
suspend/resumes.

> 3. JRuby registers to receive notifications when the VM is awakened from
> suspension (which I am assuming is possible). When it awakes, it redoes the
> computation outlined in step 1.

There is no way to know that. Not to mention that we need to detect
not only those moments when VM awakes, but also all other moments when
time changes for some reasons (daylight savings time zone, time sync
with time server, user changing the time manually, etc).

I tried various other tricks, like doing these kinds of sync-ups with
system time from time to time, but it is also tricky and unclear when
to do such things. So, at the end, there will be lots of code doing
strange/magic things that *still* won't be guaranteed to work in 100%
cases. :)

Thanks,
  --Vladimir

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Chuck R. (Guest)
on 2009-06-02 00:27
(Received via mailing list)
On Jun 1, 2009, at 3:21 PM, Vladimir S. wrote:

>> due to the
>>  # nanoTime returns a 19 digit number where the last 3 digits are
>> suspension (which I am assuming is possible). When it awakes, it
> to do such things. So, at the end, there will be lots of code doing
> strange/magic things that *still* won't be guaranteed to work in 100%
> cases. :)

Okay, sounds like there are still too many gotchas.

cr


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
This topic is locked and can not be replied to.