In this entry 7 sections
I am a CEO, and I am also an engineer. I still spend a lot of time on development. That puts me in an interesting position: I can ask my AI agent to review a feature, suggest test cases, investigate a customer problem, or draft an issue. So can a lead developer, a QA lead, or someone in support.
Lately, I have been noticing how similar the results can look.
The issues have clear titles. The descriptions sound professional. There are recommendations and acceptance criteria. Yet sometimes I finish reading and struggle to see what the person responsible for that area contributed beyond asking the agent.
That is the part that concerns me. Our expertise should change what we notice, what we investigate, and what we ask the team to do next. If those decisions disappear behind a generic answer, something valuable has gone missing from the work.
A good draft is becoming a shared starting point
There is real progress here. A colleague who struggles to explain a bug can get help writing a clear report. Someone unfamiliar with testing can discover useful cases. Support can investigate a technical question without waiting for an engineer to translate it.
Research supports that benefit. Erik Brynjolfsson, Danielle Li, and Lindsey Raymond studied an AI assistant used by 5,172 customer-support agents. Their published study found a 15% average increase in issues resolved per hour, with larger benefits for less experienced and lower-skilled workers. The most experienced and highest-skilled workers saw small speed gains and small quality declines. These were results from a particular workplace and tool, rather than a universal effect of AI. Generative AI at Work, The Quarterly Journal of Economics, 2025.
One implication is encouraging: practices that once took time to acquire can become easier to share. We should welcome that. A senior person does not need to protect their value by keeping everyone else dependent on them.
But it changes what a polished document tells us. When the tool can supply the structure and professional language, we need to look more closely at the reasoning underneath. A well-written issue may contain a verified defect, a useful hypothesis, or a plausible concern that does not apply to this product. Those deserve different treatment.
Similar answers can hide missing investigation
Imagine three people asking an agent to review an import feature. Each receives a list covering validation, error handling, performance, accessibility, and test coverage. They file the suggestions as issues.
All of those subjects could matter. The list still leaves the team with the difficult questions. Which problems actually occur? Under what conditions? Which safeguards already exist? What consequence justifies interrupting other work?
An issue is a request for someone else’s attention. When we file an unexamined recommendation, we pass the investigation downstream. Someone must discover whether the concern is real, reconstruct the context, and decide whether the proposed fix would help. A longer backlog can conceal that transfer of work.
There is a related finding in creativity research. Anil Doshi and Oliver Hauser found that access to AI-generated ideas improved evaluations of individual short stories while making the stories more similar to one another. Their experiment studied short fiction, with constrained access to AI ideas; it does not establish what happens in engineering teams. It does, however, illustrate how better individual outputs and less variety across a group can coexist. Doshi and Hauser, Science Advances, 2024.
My concern for teams is a practical inference: if everyone starts with similar instructions and accepts similar recommendations, the same omissions may survive every review. Agreement is useful when people have checked the evidence. Repetition alone gives us little additional confidence.
What each person brings to the same problem
Consider a fictional contact-import feature. A customer reports that an import timed out, then produced duplicate contacts after they retried. A generic issue might say:
Improve import error handling. Add clearer messages, automatic retries, and more tests.
That sounds reasonable. Automatic retries could also repeat the very action that created the duplicates.
Each role has a different responsibility for making the situation clearer. Open the perspectives below to see how those contributions could build on one another. The example is illustrative, not an account of an incident in my company.
“The import timed out. I tried again. Now I have duplicates.”
SupportWhat happened to the customer?
- Evidence to gather
- Read the original conversation and confirm the sequence: timeout, retry, then duplicates. Establish what the customer expected and whether earlier advice encouraged retrying.
- Judgment to contribute
- Explain the interrupted workflow and the cleanup burden. Keep the customer’s observation separate from a suspected technical cause.
QAUnder what conditions does it fail?
- Evidence to gather
- In a controlled environment, interrupt an import after some contacts are saved, then retry it. Compare the resulting records with an uninterrupted run and capture the setup and results.
- Judgment to contribute
- Define the verified scope and what remains untested. Include recovery behavior and preservation of existing records in the proposed regression checks.
EngineeringWhat mechanism explains the failure?
- Evidence to gather
- Trace the import writes and retry path. Check whether an interrupted request can leave saved records behind and whether the retry recognizes completed work.
- Judgment to contribute
- Compare safe retry and recovery options against the actual data model. Explain why a clearer message alone would leave the duplication risk unresolved.
LeadershipWhat decision should we make now?
- Evidence to gather
- Combine the verified failure, known customer impact, available mitigations, and engineering estimate. Ask which decisions still depend on missing information.
- Judgment to contribute
- Choose the immediate mitigation and release priority, name an owner, and set a checkpoint. Make the accepted risk and the reason for the decision explicit.
Together, that work supports a much more useful issue: “Retrying an interrupted import can duplicate contacts already saved.” The report can distinguish the reproduced behavior from questions still open, link the evidence, explain the customer consequence, and define what a successful fix must demonstrate.
The acceptance condition becomes concrete: retrying the same interrupted import must preserve completed work without creating duplicate contacts, and the user must be able to understand the outcome. Engineering and QA still need to agree on the exact import and retry semantics.
The difference is the investigation behind the words. Adding a job title to a prompt cannot supply a conversation nobody examined, a failure nobody reproduced, or a tradeoff nobody considered.
A capable agent may help do all of that. It can inspect code, search authorized records, run experiments, and propose a better explanation. The person responsible still needs a sound basis for accepting the result and acting on it.
Expertise should broaden with AI
I do not expect people to stay inside narrow departmental boundaries. A support colleague may identify the architectural problem. A developer may notice the customer misunderstanding. A QA lead may make the strongest product recommendation. Good evidence deserves attention wherever it comes from.
The research gives us a reason to encourage that overlap. In a field experiment with 776 professionals at Procter & Gamble, people using AI produced proposals that balanced technical and commercial considerations more evenly across professional backgrounds. Individuals with AI also matched the performance of teams without AI on the study’s product-innovation tasks. That result concerns a defined experimental setting; it does not establish that individuals can replace teams across ongoing operations. Dell’Acqua and colleagues, The Cybernetic Teammate.
So identical conclusions are not automatically a problem. Several people can independently reach the right answer. A junior colleague using AI can produce excellent work. An experienced colleague may reasonably accept an AI recommendation without changing a word.
The standard is whether the recommendation has been examined against the actual situation. Rewriting good output to demonstrate effort would be theatre. Rejecting it because a machine suggested it would be equally unhelpful.
Experience earns its place by improving decisions. It also needs to remain open to correction. “I have done this for years” should invite a useful explanation of the constraints, not end the discussion.
The value of your experience should be visible in the decisions the team can make because of your work.
Curation runs through the whole task
I have written before about how a clear brief improves AI output. That is one part of this. Professional curation continues after the first response, through investigation, selection, and follow-through.
Before asking the agent, bring what matters: the customer’s goal, the relevant history, known constraints, and the consequences of getting it wrong. Separate established facts from your own explanation. An agent can explore a hypothesis more usefully when it knows that the hypothesis still needs testing.
During the work, ask it to challenge the recommendation. Which evidence supports the suspected cause? What else could explain the behavior? Is the proposed safeguard already present? What result would make us change our minds?
This matters because strong performance on one task does not guarantee strong performance on the next. The BCG study known as Navigating the Jagged Technological Frontier found that AI improved consultants’ performance on tasks within the tested system’s capabilities, while reducing correctness on a task selected outside them. Those results concern the model and tasks studied; they are a reason to evaluate assistance in context, rather than assume a fixed boundary for today’s agents. Dell’Acqua and colleagues, Organization Science, 2026.
Before handing work over, resolve or label the uncertainty that matters. A confirmed defect, a suspected risk, and an improvement idea should be recognizable as such. Serious concerns should be raised promptly even when verification is incomplete. Clear uncertainty helps the team respond; hiding an urgent concern until the report is perfect would defeat the purpose.
After a decision, check the result. Did the fix address the customer’s problem? Did it introduce a different failure? What should become a regression test, a support note, or context for the next agent session?
That is how experience accumulates in the organization. The next person should have access to what this investigation taught us.
Review needs a purpose
“A human reviewed it” tells me very little unless I know what the review established.
A CHI 2025 survey of 319 knowledge workers found that greater confidence in generative AI was associated with less reported critical thinking. Participants also described work shifting toward verification, integrating responses, and overseeing tasks. This was self-reported evidence, so it does not prove that AI causes a loss of ability. It does make passive approval worth examining. Lee and colleagues, Microsoft Research / CHI 2025.
For a substantial issue or recommendation, I would want the author to be able to explain four things:
- What we know: the observation, reproduction, or source that supports the concern, alongside what remains uncertain.
- Why it matters here: the affected workflow, relevant constraints, and likely consequence.
- What they recommend: the next action, its priority, and the tradeoff behind it.
- How we will know: the evidence that would confirm success or change the recommendation.
These can fit in a few sentences. They are prompts for thinking, not a requirement to make every small bug report into a research paper. The depth should match the consequence of being wrong.
Useful review can also be delegated to tools and automated checks. We should choose checks because they can catch the relevant failure, then examine exceptions and unresolved risks. Having a person reread everything without a clear purpose can become another expensive ritual.
Leadership has to reward this work
As a CEO who also builds, I need to apply the same standard to myself. My agent can produce an impressive review. My title does not make that review better informed than a QA lead’s investigation or a support colleague’s direct knowledge of the customer.
If I turn every suggestion from my agent into an assignment, I can create the same problem at a larger scale. People may stop challenging a weak recommendation because it arrived from the CEO. I need to make it clear when I am offering a question, sharing a hypothesis, or making a decision—and explain the basis for the decision.
I also need to leave room for specialists to own outcomes. Being able to generate work across several disciplines does not give me the attention to investigate every detail, maintain every relationship, and follow every result. A team becomes more capable when decisions can be made well without returning to me each time.
The incentives matter. If we celebrate issue counts, documents produced, and visible activity, we should expect more of them. We should also recognize the person who disproves a misleading concern, combines duplicate reports into one diagnosis, catches a release risk, or saves a colleague a day of investigation.
Those contributions are sometimes quieter. They make the next decision easier.
I want people to use AI deeply. I want them to move beyond tasks their job title once limited them to. I also want their contact with the product, the code, the customers, and previous failures to shape what the team receives.
When I read a piece of work, I should be able to understand what we learned, why it matters, and what its author is prepared to recommend.
That is where I expect our expertise to show.
