Time and microseconds

cremes · June 1, 2009, 5:34pm

Charlie,

we chatted about this briefly on irc but I guess I don’t understand
why the Time class doesn’t generate microseconds. I’ve been calling
System.nanoTime and adding it to my calls to Time.now and it has been
pretty good. Why wouldn’t a patch like this work?

— RubyTime.java 2009-05-29 12:26:32.000000000 -0500
+++ RubyTime.java.new 2009-06-01 10:28:42.000000000 -0500
@@ -169,7 +169,7 @@
DateTimeZone dtz = getLocalTimeZone(runtime);
DateTime dt = new DateTime(dtz);
RubyTime rt = new RubyTime(runtime, klass, dt);

```
       rt.setUSec(0);
```

       rt.setUSec(((System.nanoTime() / 1000) % 1000000) /

1000000);

          return rt;
      }

If I recall, you mentioned that suspending/sleeping the VM caused
System.nanoTime to freak out. Am I remembering correctly?

cr

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 5:50pm

Just saw this:

http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6480016

Specifically it states in the evaluation:

The specification of nanoTime is very clear that it
does not bear any relation to absolute time:

“This method can only be used to measure elapsed time and is not related
to any other notion of system or wall-clock time. The value returned
represents nanoseconds since some fixed but arbitrary time (perhaps in
the future, so values may be negative).”

See also
http://blogs.sun.com/dholmes/entry/inside_the_hotspot_vm_clocks
Posted Date : 2006-10-10 18:08:29.0

Hope that helps,
Gary

Chuck R. wrote:

         DateTimeZone dtz = getLocalTimeZone(runtime);
System.nanoTime to freak out. Am I remembering correctly?

cr

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 5:54pm

Chuck R. wrote:

         DateTimeZone dtz = getLocalTimeZone(runtime);
         DateTime dt = new DateTime(dtz);
         RubyTime rt =  new RubyTime(runtime, klass, dt);

```
       rt.setUSec(0);
```

       rt.setUSec(((System.nanoTime() / 1000) % 1000000) / 1000000);

       return rt;
   }

This is actually setting it to whatever nanTime happens to be at any
given moment, which will be continuously increasing. It’s not really a
valid nano/usec component of the current time unless you’re tracking it
from a beginning point, which is where we run into problems.

If I recall, you mentioned that suspending/sleeping the VM caused
System.nanoTime to freak out. Am I remembering correctly?

That is correct; nanotime does not increment while a machine/VM is
sleeping, so you get peculiar results if it is the only thing used for
calculating time.

I know Tom would like to get sub-ms precision back into Time, so we’re
open to finding a patch that does that without having suspend issues.

Charlie

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 6:31pm

Hi Chuck,

On Mon, Jun 1, 2009 at 5:33 PM, Chuck R. [email protected]
wrote:

If I recall, you mentioned that suspending/sleeping the VM caused
System.nanoTime to freak out. Am I remembering correctly?

I tried really hard to implement proper microseconds in JRuby’s Time,
and for a while I thought that it worked, but at the end, way too much
magic and incorrect work in virtual environments and during the time
shifts (winter->summer time, etc). So now I’m convinced that it is
close to impossible to implement proper microseconds in pure-Java (via
nanoSeconds()). nanoSeconds specifically doesn’t deal with Time,
only with time intervals, and it cannot be used to construct Time
objects.

The only practical solution would probably be to use JFFI to call
underlying platform API to obtain the time with needed precision. The
major drawback here is that such implementation would probably slower.

Thanks,
–Vladimir

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 6:19pm

On Jun 1, 2009, at 10:54 AM, Charles Oliver N. wrote:

        DateTime dt = new DateTime(dtz);
tracking it from a beginning point, which is where we run into
we’re open to finding a patch that does that without having suspend
issues.

I see. Gary’s response clued me in.

I’ll see if I can’t come up with a solution to get back microseconds.

cr

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 6:47pm

Found a thread from 2006 stating some of the (what seem to be many)
issues with nanoTime():
http://www.velocityreviews.com/forums/t151271-p-systemnanotime-and-multiple-cpuscores.html

like supposedly:

System.nanoTime() uses QueryPerformanceCounters() on Windows and that
this function is known to have problems on Athlon64 multicore systems

Also here:

which states:

“During my investigations I also found out
that on Windows platforms some chipsets cause leaps forward in time (up
to a
few seconds) when using the QueryPerformanceCounter-Call (which is -to
my
knowledge- the only way to measure in nanoseconds on Windows). This
behavior is
documented under:”

Microsoft Support;

and that says:

“Programs should watch for an unexpected jump by comparing the change in
time as determined by successive calls to QueryPerformanceCounter with
the change in time as determined by successive calls to the GetTickCount
function. If there is a significant jump that is based on
QueryPerformanceCounter(), but no similar increase that is based on
GetTickCount, then it can be assumed that the performance counter just
jumped forward. The code sample at the end of this article demonstrates
how to do this. This operating system’s behavior is by design. The
performance counter adjustment is necessary when the operating system
obtains unreliable data from the chipset.”

I just downloaded the latest Java 1.6 source
(openjdk-6-src-b16-24_apr_2009) and there are three occurrences of
QueryPerformanceCounters() in hotspot/src/os/windows/vm/os_windows.cpp
and one in jdk/src/windows/bin/java_md.c.

You got to love the comment “the best we can do”, which appears to
describe the issue Microsoft described above:

hotspot/src/os/windows/vm/os_windows.cpp-#define NANOS_PER_SEC
CONST64(1000000000)
hotspot/src/os/windows/vm/os_windows.cpp-#define NANOS_PER_MILLISEC
1000000
hotspot/src/os/windows/vm/os_windows.cpp-jlong os::javaTimeNanos() {
hotspot/src/os/windows/vm/os_windows.cpp- if (!has_performance_count) {
hotspot/src/os/windows/vm/os_windows.cpp- return javaTimeMillis() *
NANOS_PER_MILLISEC; // the best we can do.
hotspot/src/os/windows/vm/os_windows.cpp- } else {
hotspot/src/os/windows/vm/os_windows.cpp- LARGE_INTEGER
current_count;
hotspot/src/os/windows/vm/os_windows.cpp:
QueryPerformanceCounter(&current_count);
hotspot/src/os/windows/vm/os_windows.cpp- double current =
as_long(current_count);
hotspot/src/os/windows/vm/os_windows.cpp- double freq =
performance_frequency;
hotspot/src/os/windows/vm/os_windows.cpp- jlong time =
(jlong)((current/freq) * NANOS_PER_SEC);
hotspot/src/os/windows/vm/os_windows.cpp- return time;
hotspot/src/os/windows/vm/os_windows.cpp- }
hotspot/src/os/windows/vm/os_windows.cpp-}

In addition, here is the javadoc comment in the latest 1.6 source
(openjdk-6-src-b16-24_apr_2009) (I could have gotten from API docs
somewhere, but I was in there anyway) which notes an issue with
nanoTime() with spans of time greater than 292 years (2^63 nanoseconds):

/**
 * Returns the current value of the most precise available system
 * timer, in nanoseconds.
 *
 * <p>This method can only be used to measure elapsed time and is
 * not related to any other notion of system or wall-clock time.
 * The value returned represents nanoseconds since some fixed but
 * arbitrary time (perhaps in the future, so values may be
 * negative).  This method provides nanosecond precision, but not
 * necessarily nanosecond accuracy. No guarantees are made about
 * how frequently values change. Differences in successive calls
 * that span greater than approximately 292 years (2<sup>63</sup>
 * nanoseconds) will not accurately compute elapsed time due to
 * numerical overflow.
 *
 * <p> For example, to measure how long some code takes to execute:
 * <pre>
 *   long startTime = System.nanoTime();
 *   // ... the code being measured ...
 *   long estimatedTime = System.nanoTime() - startTime;
 * </pre>
 *
 * @return The current value of the system timer, in nanoseconds.
 * @since 1.5
 */
public static native long nanoTime();

Hope that helps!

Gary

Chuck R. wrote:

+++ RubyTime.java.new 2009-06-01 10:28:42.000000000 -0500
This is actually setting it to whatever nanTime happens to be at any

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 8:48pm

On Jun 1, 2009, at 11:31 AM, Vladimir S. wrote:

magic and incorrect work in virtual environments and during the time
shifts (winter->summer time, etc). So now I’m convinced that it is
close to impossible to implement proper microseconds in pure-Java (via
nanoSeconds()). nanoSeconds specifically doesn’t deal with Time,
only with time intervals, and it cannot be used to construct Time
objects.

The only practical solution would probably be to use JFFI to call
underlying platform API to obtain the time with needed precision. The
major drawback here is that such implementation would probably slower.

I have an idea. Let me know if this sounds nuts.

When the RubyRuntime is instantiated, save the current
System.nanoTime as a “nano epoch” and do the “expensive” jni call to
the underlying platform and save that as a “platform epoch”. There
will be slippage here due to the call overhead but we’ll assume the
call takes constant time.

I assume get_time_of_day returns a 19 digit number to represent

nanoseconds just

like nanoTime does

nano_epoch = System.nanoTime
platform_epoch = expensive_call_to_get_time_of_day

round the platform time down to the nearest second

usec_remainder = (platform_epoch % 1000000000) / 1000

sets nanoTime epoch to be closest to the nearest second

relative to the platform’s time

nano_epoch -= usec_remainder

Calls to Time.now will populate the +usec+ field by doing a simple
calculation.

def get_usec

nanoTime returns a 19 digit number where the last 3 digits are

always 0
usec = ((System.nanoTime - nano_epoch) % 1000000000) / 1000
end

JRuby registers to receive notifications when the VM is awakened
from suspension (which I am assuming is possible). When it awakes, it
redoes the computation outlined in step 1.

Or am I crazy that this will work?

cr

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 6:51pm

2009/6/2 Vladimir S. [email protected]:

The only practical solution would probably be to use JFFI to call
underlying platform API to obtain the time with needed precision. The
major drawback here is that such implementation would probably slower.

If you want to see how much slower, try running
bench/ffi/bench_gettimeofday.rb in 1.3.0RC2 or 1.3.0 on anything but
windows.

That is using the ruby FFI interface, but it’ll be similar to the java
version. On my machine the gettimeofday call is about 60% slower than
Time.now.

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 9:08pm

Chuck R. wrote:

I have an idea. Let me know if this sounds nuts.

When the RubyRuntime is instantiated, save the current
System.nanoTime as …

and I think that is where it breaks down, because System.nanoTime is not
guaranteed to be accurate on all platforms. Even if everything after
that point was ok, it would continuously give an inaccurate number if
the number was based on nanoTime.

That’s not from what I read- I’m not claiming to be an expert on it.
Read previous mails for more info as to why.

Hope that helps,

Gary

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 9:22pm

On Jun 1, 2009, at 2:07 PM, Gary W. wrote:

and I think that is where it breaks down, because System.nanoTime is
not guaranteed to be accurate on all platforms. Even if everything
after that point was ok, it would continuously give an inaccurate
number if the number was based on nanoTime.

That’s not from what I read- I’m not claiming to be an expert on it.
Read previous mails for more info as to why.

I understand that. So we have three choices.

Detect the unreliable platforms and use the current scheme that is
accurate to the millisecond. Otherwise, use my mechanism or whatever
works.
Use the newer scheme to generate microseconds with a documented
caveat that they may be inaccurate on specific platforms.

#1 is probably the better choice. Use the greater precision where it
can be trusted with a fallback mode.

cr

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 10:22pm

On Mon, Jun 1, 2009 at 8:46 PM, Chuck R. [email protected]
wrote:

Calls to Time.now will populate the +usec+ field by doing a simple
calculation.

def get_usec

nanoTime returns a 19 digit number where the last 3 digits are always 0

usec = ((System.nanoTime - nano_epoch) % 1000000000) / 1000
end

That’s how it was implemented some time ago in JRuby, and it worked
most of the time, but was breaking during time shifts and VM
suspend/resumes.

JRuby registers to receive notifications when the VM is awakened from
suspension (which I am assuming is possible). When it awakes, it redoes the
computation outlined in step 1.

There is no way to know that. Not to mention that we need to detect
not only those moments when VM awakes, but also all other moments when
time changes for some reasons (daylight savings time zone, time sync
with time server, user changing the time manually, etc).

I tried various other tricks, like doing these kinds of sync-ups with
system time from time to time, but it is also tricky and unclear when
to do such things. So, at the end, there will be lots of code doing
strange/magic things that still won’t be guaranteed to work in 100%
cases.

Thanks,
–Vladimir

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 9:09pm

Gary W. wrote:

That’s not from what I read

I meant “That from what I read”. Gees- I need more sleep!

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

cremes · June 1, 2009, 10:27pm

On Jun 1, 2009, at 3:21 PM, Vladimir S. wrote:

due to the

nanoTime returns a 19 digit number where the last 3 digits are

suspension (which I am assuming is possible). When it awakes, it
to do such things. So, at the end, there will be lots of code doing
strange/magic things that still won’t be guaranteed to work in 100%
cases.

Okay, sounds like there are still too many gotchas.

cr

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email