Cisco is known for the rock-solid reliability and amazing technical support. About the only complaints we ever hear about Cisco is they tend to be on the high end of pricing, which is can be true when you don’t have a partner like UCRight who will pass long the heavy discounts, and that they can sometimes be a challenge to program or configure without a GUI.
It’s not uncommon for our engineers to enter a client’s server room, or datacenter, and see Cisco gear that is a decade old still happily humming along and doing it’s job. Nor is it an uncommon refrain to hear from a client that they stopped paying for Smartnet, or just didn’t care that they could no longer buy it due to EOL/EOS, because the equipment “just works”.
However, as we saw with Dell two decades ago and their infamous ‘capacitor plague’, wiki it if you don’t recall, or as other vendors over the years have experienced in the past…. sometimes things are out of your control and can still have a dramatic impact on your bottom line/reputation. But how a company handles these issues is what really makes the difference both to its customers and its brand.
This is why we, at UCRight, were impressed by the acknowledgement and ongoing measures Cisco has put into place to handle Cisco’s “Clock Signal Component Issue”.
The issue first discovered in 2016 and later made public that same year – can be summarized as follows:
A downstream provider, Cisco will not say which one or if they continue to work with them, produced components for Cisco devices which in this specific case have found to be faulty or defective. The component in question is the “Clock Signal Component”, which is found in many of Cisco’s products produced during this time including; Meraki, Nexus, Cisco ISR (routers), Adaptive Security Devices (ASA), and more…the full list is here.
The defect or bug in this case is that over time the component will degrade and then eventually completely fail. The time line for this is 18 months after the device was placed into service (when you started using it), although Cisco has indicated it expects the highest level of failures to occur at 24-36 months.
There is no field serviceable solution, or resolution, meaning the only thing that can be done is to replace the device. If left unaddressed and a device is impacted, this failure will lead to a completely ‘bricked’ device (i.e. dead, no recovery, no backup, no unplug and hope a reboot fixes, etc).
However, like we indicated above, Cisco has impressed us by taking the high road on this one. What they have said is that they will replace any impacted device that was covered by a warranty or SmartNet as of 11/16/2016. We felt this was a much better tack to take then other vendors in the past that have essentially said “oh, you’re not covered by a support contract currently? Too bad, sounds like you need to buy a new device”.
Further, Cisco’s transparency in this matter has been commendable. Whereas with some vendors you have to login to their site to get info like this (essentially treating it as proprietary), or go scouring the web for the info, Cisco has created a central repository to address the problem, a concise list to identify the devices impacted and a detailed FAQ answer many of the questions partners, and clients alike, may have.
While I am not going to delve much deeper into the weeds on this one, I do want to point out three things that we have found some confusion over:
- The onus of getting this addressed is on the end user who owns the device; or in a perfect world on the partner who sold it to them. This means you need to look at your own devices, or have a partner do it for you, and see if they are impacted and in need of replacement. Cisco will not magically show up at your door step with a new device.
- Cisco will not provide professional services to help you swap the faulty device with the replacement. This will have to be handled by the end user or by their preferred integration partner (UCRight has been doing this for free for any client we sold one of these devices too).
- The order in which Cisco is replacing devices is determined by how long ago they were sold, or put into service. Essentially, they are using ‘triage’ method – making sure the most likely to fail (from a time perspective) are addressed first. So as an example – impacted products that were the last to be built with the faulty “clock signal component” will also likely be the last to be replaced.
TL;DR – if you think you might be impacted by this bug and have not heard from your own vendors…..then time is of the essence for you to get it addressed.