NBE Agent Fleet

I embedded with a small operations team running the manual side of digital engagement campaigns: processing instruction files, auditing delivery, tracking execution, and reprocessing broken jobs. The work depended heavily on tribal knowledge, so I spent months learning the workflows directly from the people doing them, then turned the highest-friction processes into reusable tools and agents. This was the first agent system I built, and most of what I now know about running agents against production systems I learned here.

What I shipped

Nine automations over about a year, including:

• A GUI-based reprocessing platform that let non-technical teammates run end-to-end file reprocessing without a terminal.
• An engagement tracking agent connected to Google Ad Manager that reduced a multi-hour manual workflow to about 60 seconds.
• A Tableau/MCP waterfall reporting agent that replaced more than 30 manual spreadsheets with a single shared view.
• A triage agent that monitored incoming files daily against a Google Drive knowledge base and flagged missing items proactively.
• A compliance validation agent connected to Salesforce, Google Sheets, and Tableau to catch missing deliverables before they became issues.
• A Pentaho audit QA agent that removed a 15-hour-per-week email-driven process and enabled analysts to run it themselves.
• A UAT agent that validated client setups before launch using Jira, Gmail, and test-file generation.
• An audit tracking MVP enhanced by agents that searched Drive and email to populate status automatically.
• A daily monitoring dashboard that gave the team a single operational view across incoming files, queue position, errors, and escalation triggers.

The connectors

Seven of the nine ran in the Gemini web app as Gems, with no self-hosted infrastructure. To reach systems Gemini couldn't see, I wrote MCP servers for Google Ad Manager, Jira, and Tableau. Each exposed the read and write operations a specific workflow needed rather than wrapping the full API. Auth was API-key based, running inside the corporate network over VPN, with nothing publicly reachable. That was the right scope for internal pilots and the wrong one for scale, since the keys were mine, so every user's access inherited my permissions instead of their own. A service account with its own scoped role is what I'd build now. The org later restricted custom connectors org-wide and the servers were retired.

Results

More importantly, the systems were documented and handed off so they could be run without me, including during a staffing gap.

How I checked correctness

Each agent was validated against a hand-built golden set: fixed input cases with known-good outputs, reviewed manually with the workflow owner before hand-off. Scoring was our judgment of whether the output was right, not an automated harness.

What broke

The model was rarely the problem. The first failure was mine: the agent pulled the wrong campaign data because I had specified the task badly. That was a prompt fix, done in an afternoon.

The failures that mattered were environmental. Google Ad Manager rate-limits requests, so a tool the agents depended on was not always available when they asked for it. Tools also timed out mid-workflow, which is worse than failing outright, because some steps had committed and some had not and nothing in the system knew which.

I handled both with monitoring rather than engineering. The agents surface an error when a run does not complete, and Jaidev, the analyst I trained on the system, re-runs it. That comes to a couple of re-runs a month across nine automations. For one person shipping nine automations in a year against a team that needed the hours back immediately, monitored recovery was the right call, and it kept a person watching a system nobody had run before.

What I'd do differently

Three things, all of which I would build now and did not know to build then. Batch and cache requests against the rate limit so the agents work within the quota instead of hitting it. Make each step idempotent, so a re-run cannot double-process a file that already went through. Checkpoint the workflow, so a timeout resumes from the last completed step instead of restarting.

None of these are model problems. They are the failure modes you only see once agents are running against production systems on a schedule, and they are what I look for first now.

Stack