Hi I had this kind of problem many times.
Usually problem is because of bottleneck in the transport network causing high latency.
You can try to ping the Ipsync then see if there's high latency.
IEEE 1588 is sensitive to PDV.
If you have high latency but constrained PDV then no issue exists. The problems start when excessive PDV (Packet Delay Variation) occurs.
Bookmarks