An organization's overall effectiveness score says it is performing well, while its own leaders rate most projects as troubled and not one pilot participant calls the intake process healthy.
Not a failing organization
The global IT organization supported nearly everything its company did: sales, recruiting, payroll, customer platforms, and the internal services that kept the business running. By most measures it was performing well. When we ran an organizational effectiveness survey across the whole IT function at the start of the work, it scored 81.1 percent.
That number deserved respect. This was not a failing organization. It was a capable one, staffed by people doing good work. It was also exactly the kind of number that lets leadership relax, and that is the danger in it. An overall score that high says the organization is broadly fine. It does not say where it is not.
The survey itself is honest about what it measures. It captures how effective people perceive the organization to be: how well it strategizes, prioritizes, delivers, and adapts under pressure. It does not measure any particular operational result. A healthy perception and a broken process can sit side by side, and here they did.
Two other readings of the same organization
The IT leadership team rated its own recent projects against a standard definition of success: on time, on budget, and with a satisfied customer. By their own count, 35.4 percent of projects met that bar. That beat the industry benchmark, and it still meant most projects ran into trouble somewhere along the way.
Then the pilot group rated its own intake process, step by step, as healthy, at risk, or critical. Not one person rated it healthy. Half the participants called it critical; the other half called it at risk. Work was arriving through six channels, formal and informal, and much of it stayed invisible to leaders until it was already consuming their teams' time.
None of these numbers contradicted the others. Each answered a different question. The effectiveness score answered how people experienced the organization as a whole. The project ratings answered how often delivery actually landed. The intake ratings answered whether the front door worked. The headline was not lying. It was averaging away the conditions that most needed attention, and five of those conditions, from intake to reporting, were feeding one another underneath it.
So how will we know it is working?
Later, when the CIO saw the full plan for the pilot laid out in one place for the first time, he asked the question that decided how the rest would be judged. How would they know it was working?
Our answer was five dimensions, not one score. Readiness would show whether the conditions for success were in place. Business value and satisfaction would show whether stakeholders believed the work was creating value. Visibility and standardization, intake and prioritization, and release planning and execution would each show whether a specific operational change was taking hold. Measuring only business outcomes would say whether things improved without saying why. Measuring only operations would say what changed without saying whether the business valued it.
The CIO saw the point quickly. They were not looking for a single score. We were not looking for a rubber stamp. Organizations often search for one number that proves success or failure, and complex systems rarely work that way. A rising satisfaction score can hide deteriorating operational performance. Faster delivery can mask declining quality. Strong adoption can disguise weak business outcomes. The only way to see what is really happening is to look from several angles at once.
| Question it answers | Example measure | |
|---|---|---|
| Readiness | Are conditions for success present? | Active change champions |
| Value and satisfaction | Does the business feel served? | Perceived value delivered |
| Visibility | Can people see across teams? | Use of shared dashboards |
| Intake and priorities | Is the new process really used? | Share of work through the forum |
| Releases | Does delivery land predictably? | Lead time for requests |

Measures built to catch each other
Each dimension was built so that its measures could check one another. Readiness came first, because it measured starting conditions rather than results: a self-assessment by the pilot group, the number of active change champions, and a leadership checkpoint on whether the pilot still served the strategy. Outcome measures reveal problems after the damage is done. These were meant to show whether success was becoming more or less likely while there was still time to adjust.
When a team member warned that a satisfaction score would swing with whoever's project had just slipped, the answer was not to drop it. It was to sample it quarterly and pair it with a measure of perceived value, which was less sensitive to the latest interaction. The CIO put the logic in his own terms: if satisfaction moved and perceived value did not, that meant something different than if both moved together.
In intake and prioritization, one measure tracked the share of active work that had actually passed through the new priority forum. A member of the internal change team called it the most important number on the board. If people were not using the forum, the other measures would mean little, and the organization could end up running the old system under a new name. When the CIO asked what the baseline was for the time between intake and the start of work, the honest answer was that it did not exist yet. Establishing it became part of the first phase of the plan, before anyone set a target.
A better score, read honestly
When the pilot closed, the effectiveness survey ran again and came back at 83.1 percent, with the clearest gains in clarity of direction and confidence in decision-making. That reading needs its scope stated plainly. The first survey covered all of IT; the second covered only the pilot team. It speaks for the pilot, not for the whole function.
The other readings carried the substance. Six ways for work to come in had become two. Five hand-built reports had become three dashboards that update themselves. Priority calls were being made against shared criteria and written down.
And the set of measures kept doing the job it was designed for. Even with the headline score up, getting the resources a team needed was still the weakest area the pilot group reported. Leaders working visibly in the new system remained the open item that mattered more than any further design change. A single number would have said the work was done. The readings underneath said exactly where it was not, which is the only kind of information anyone can act on.
One healthy number tells you how the organization feels on average. The readings underneath tell you where it is failing, and only those can be acted on.
Share With Your Team
Choose who it is for. You send it yourself from your own email; Agileon never writes to your colleagues.


