I hear this lament all too often from boards and CEOs. Knowing when to pull the trigger on executive changes is typically very difficult and it is just as frequently a decision that, in retrospect, many will say they should have made "months ago". Knowing that Henry Kravis struggles with this should provide some solace, yet it still doesn't make the decision any easier. Eliminating emotion from business decisions is typically good advice, yet this is one business decision where that is much easier said than done. Also - CEO and other executive team changes are highly disruptive to the organization, so our natural reflex is to try to make do in order to avoid that disruption.
In an interview with Kip McDaniel at Institutional Investor, Kravis was asked to identify a mistake he repeated during his 40 years at KKR. His answer was pretty brutal, but it gets to a classic Wall Street concept: knowing when to cut your losses. Here's what Kravis had to say: "We might have been too slow in changing out some CEOs of companies we had, keep thinking that he or she will get a lot better. I can pretty well tell you that what you see up front is pretty much what you'll see in the end. You can help around the edges, but people don't change that much.
http://www.businessinsider.com/henry-kravis-on-the-mistake-he-made-again-and-again-2017-2