VDS 2008 R2 64bit - Time Sychronization Problem

[Oaker]

I've never run across something like this before... so any help would be greatly appreciated.

I have been experiencing an issue where time will randomly de-synch. It typically advances anywhere from 1 hour to over 24 hours. I've never seen a system fall out of synch to such a degree. It is random, can be daily or several times a day, or skip a day or two. I'm baffled.

I applied the regedit via below link and the system still falls out of time.
http://www.nfoservers.com/forums/viewto ... =44&t=8658

There's nothing exotic running on this machine.... game servers.

I've tried windows.time, a-nist, nist, etc. Firewall, no firewall.

Manual synch (cmd & gui) typically works, but not always on the first or second try. Another odd thing is sometimes the gui interface on Internet Time is blank...?

Below is a typical event.

Warning 1/25/2013 2:05:52 PM Time-Service 36 None

Log Name: System
Source: Microsoft-Windows-Time-Service
Date: 1/25/2013 2:05:52 PM
Event ID: 36
Task Category: None
Level: Warning
Keywords: Classic
User: N/A
Computer: tenthvps
Description:
The time service has not synchronized the system time for 86400 seconds because none of the time service providers provided a usable time stamp. The time service will not update the local system time until it is able to synchronize with a time source. If the local system is configured to act as a time server for clients, it will stop advertising as a time source to clients. The time service will continue to retry and sync time with its time sources. Check system event log for other W32time events for more details. Run 'w32tm /resync' to force an instant time synchronization.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Time-Service" Guid="{06EDCFEB-0FD0-4E53-ACCA-A6F8BBF81BCB}" EventSourceName="W32Time" />
<EventID Qualifiers="33370">36</EventID>
<Version>0</Version>
<Level>3</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2013-01-25T20:05:52.000Z" />
<EventRecordID>713143</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>System</Channel>
<Computer>tenthvps</Computer>
<Security />
</System>
<EventData Name="MSG_TIME_SOURCE_NONE">
<Data Name="UnsynchronizedTimeSeconds">86400</Data>
</EventData>
</Event>

[kraze]

Did you try setting up a scheduled task also described in the link in the other thread?

http://www.pretentiousname.com/timesync/index.html

[Oaker]

Here is the Chat Log from one of our servers with time.,,, it jumps 24 hours and 10 minutes.
Saturday, 26/01/2013 (15:09) Loekie1972 (ALL) lol
Saturday, 26/01/2013 (15:09) AFISHINGLURE (ALL) hehehhe
Saturday, 26/01/2013 (15:09) [SsC]StressOF (ALL) spastis was shot
Saturday, 26/01/2013 (15:08) [SsC]StressOF (ALL) woa dodged that zuke
Saturday, 26/01/2013 (15:08) [SsC]StressOF (ALL) nooo allure
Saturday, 26/01/2013 (15:07) [OFL]EgzeqtoR (ALL) sry
Saturday, 26/01/2013 (15:07) AFISHINGLURE (ALL) ns pear
Friday, 25/01/2013 (14:57) [10th]echo4papa (ALL) !s rule5
Friday, 25/01/2013 (14:57) AFISHINGLURE (ALL) stop throwing blind nades into smoke
Friday, 25/01/2013 (14:55) [SsC]StressOF (ALL) don't stay there too long stan
Friday, 25/01/2013 (14:54) Gilgrape (ALL) np
Friday, 25/01/2013 (14:54) Gilgrape (ALL) sorry
Friday, 25/01/2013 (14:54) [10th]iccess2 (ALL) !w gil crouch
Friday, 25/01/2013 (14:53) [10th]iccess2 (ALL) gilgrape crouch please


Here's the Audit record from the previous posts Event.

Audit Success 1/25/2013 2:05:52 PM Microsoft Windows security auditing. 4616 Security State Change

Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 1/25/2013 2:05:52 PM
Event ID: 4616
Task Category: Security State Change
Level: Information
Keywords: Audit Success
User: N/A
Computer: tenthvps
Description:
The system time was changed.

Subject:
Security ID: LOCAL SERVICE
Account Name: LOCAL SERVICE
Account Domain: NT AUTHORITY
Logon ID: 0x3e5

Process Information:
Process ID: 0x1e0
Name: C:\Windows\System32\svchost.exe

Previous Time: 2:05:52 PM 1/25/2013
New Time: 2:05:52 PM 1/25/2013

This event is generated when the system time is changed. It is normal for the Windows Time Service, which runs with System privilege, to change the system time on a regular basis. Other system time changes may be indicative of attempts to tamper with the computer.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-a5ba-3e3b0328c30d}" />
<EventID>4616</EventID>
<Version>0</Version>
<Level>0</Level>
<Task>12288</Task>
<Opcode>0</Opcode>
<Keywords>0x8020000000000000</Keywords>
<TimeCreated SystemTime="2013-01-25T20:05:52.758Z" />
<EventRecordID>1061059</EventRecordID>
<Correlation />
<Execution ProcessID="660" ThreadID="712" />
<Channel>Security</Channel>
<Computer>tenthvps</Computer>
<Security />
</System>
<EventData>
<Data Name="SubjectUserSid">S-1-5-19</Data>
<Data Name="SubjectUserName">LOCAL SERVICE</Data>
<Data Name="SubjectDomainName">NT AUTHORITY</Data>
<Data Name="SubjectLogonId">0x3e5</Data>
<Data Name="PreviousDate">1/25/2013</Data>
<Data Name="PreviousTime">2:05:52 PM</Data>
<Data Name="NewDate">1/25/2013</Data>
<Data Name="NewTime">2:05:52 PM</Data>
<Data Name="ProcessId">0x1e0</Data>
<Data Name="ProcessName">C:\Windows\System32\svchost.exe</Data>
</EventData>
</Event>

[Oaker]

Did you try setting up a scheduled task also described in the link in the other thread?

http://www.pretentiousname.com/timesync/index.html

I did... with one exception, I did not select highest priority.

I correct that now.

[Oaker]

Got this in responce to a win32tm /tz command.

Problem signature:
Problem Event Name: APPCRASH
Application Name: w32tm.exe
Application Version: 6.0.6001.18000
Application Timestamp: 47919586
Fault Module Name: StackHash_43c8
Fault Module Version: 6.0.6002.18541
Fault Module Timestamp: 4ec3e855
Exception Code: c0000374
Exception Offset: 00000000000acb17
OS Version: 6.0.6002.2.2.0.272.36
Locale ID: 1033
Additional Information 1: 43c8
Additional Information 2: 7afb45e905daaea53e9fde9fbcd88b7d
Additional Information 3: c33d
Additional Information 4: d90c4059b89530e803820982645bbc4e

Read our privacy statement:
http://go.microsoft.com/fwlink/?linkid= ... cid=0x0409

[Oaker]

I have the scheduled task in place... fingers crossed.

[Oaker]

The events below show a 36 hour leap forward in time/date. The task scheduled (as described in previous link) was performing well until this event and then began failing.

I am stumped...

- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
<Provider Name="Microsoft-Windows-TaskScheduler" Guid="{de7b24ea-73c8-4a09-985d-5bdadcfa9017}" />
<EventID>201</EventID>
<Version>0</Version>
<Level>4</Level>
<Task>201</Task>
<Opcode>2</Opcode>
<Keywords>0x8000000000000000</Keywords>
<TimeCreated SystemTime="2013-01-30T06:25:15.160Z" />
<EventRecordID>77241</EventRecordID>
<Correlation ActivityID="{159D29C5-D6C7-4AD0-90E7-FA7705D58F49}" />
<Execution ProcessID="6868" ThreadID="3688" />
<Channel>Microsoft-Windows-TaskScheduler/Operational</Channel>
<Computer>tenthvps</Computer>
<Security UserID="S-1-5-21-2877598006-2160316611-2584817644-500" />
</System>
- <EventData Name="ActionSuccess">
<Data Name="TaskName">\Time Sync</Data>
<Data Name="TaskInstanceId">{159D29C5-D6C7-4AD0-90E7-FA7705D58F49}</Data>
<Data Name="ActionName">C:\Windows\system32\w32tm.exe</Data>
<Data Name="ResultCode">0</Data>
</EventData>
</Event>

Followed an hour later by...

- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
<Provider Name="Microsoft-Windows-TaskScheduler" Guid="{de7b24ea-73c8-4a09-985d-5bdadcfa9017}" />
<EventID>201</EventID>
<Version>0</Version>
<Level>4</Level>
<Task>201</Task>
<Opcode>2</Opcode>
<Keywords>0x8000000000000000</Keywords>
<TimeCreated SystemTime="2013-02-01T18:25:00.709Z" />
<EventRecordID>77330</EventRecordID>
<Correlation ActivityID="{6FB920CB-23E0-4956-A135-EBBF35F43AB4}" />
<Execution ProcessID="1236" ThreadID="2572" />
<Channel>Microsoft-Windows-TaskScheduler/Operational</Channel>
<Computer>tenthvps</Computer>
<Security UserID="S-1-5-21-2877598006-2160316611-2584817644-500" />
</System>
- <EventData Name="ActionSuccess">
<Data Name="TaskName">\Time Sync</Data>
<Data Name="TaskInstanceId">{6FB920CB-23E0-4956-A135-EBBF35F43AB4}</Data>
<Data Name="ActionName">C:\Windows\system32\w32tm.exe</Data>
<Data Name="ResultCode">0</Data>
</EventData>
</Event>

[Edge100x]

If syncing the time caused it to be radically wrong, it could be a problem with the default Windows time source. Try changing it to time.nist.gov or bigben.cac.washington.edu or something different.

[Oaker]

I have tried "nist" and "a-nist".... I'll try another....

[Edge100x]

A Xen kernel developer posted a patch to correct a bug that may have caused this time skew issue. We're currently testing to see if it fixes the problem.

[Oaker]

A Xen kernel developer posted a patch to correct a bug that may have caused this time skew issue. We're currently testing to see if it fixes the problem.

Hope it proves workable... as this issue continues with our new WS 2012 install.

My fingers and toes are crossed...

[Oaker]

In this recent event the clock jumped forward about 4 hours and 10 minutes. I do not recall noticing mention of hardware clock in previous events.

The system time has changed to ?2013?-?02?-?11T02:17:22.500000000Z from ?2013?-?02?-?10T22:03:23.691988900Z.

Change Reason: System time synchronized with the hardware clock.

[Edge100x]

Oaker, please open a support request and I can schedule a maintenance event to bump up the version of Xen on your machine to the patched one.

[Oaker]

Oaker, please open a support request and I can schedule a maintenance event to bump up the version of Xen on your machine to the patched one.

Will do... thanks.