The second audit
Earlier this month, a security audit of Groundworks AI found two high-severity problems in the identity lifecycle around account email changes and team-member access. We remediated both, added regression tests and passed the full build. Those two findings are closed.
Then I asked for another audit. The next review found new issues that required immediate remediation in the production service. That can sound like the first audit failed. I think it means the process did its job.
A security audit is not a certificate you earn. It is a point-in-time claim that has to survive new code, new architecture, new dependency advisories and the reality of the deployed system. If you do not run it again, it quietly turns into old reassurance.
Why clean reports expire
The security state of software can change even when its code does not. A dependency audit that was green on 11 July can be red on 31 July because new vulnerabilities were disclosed in between. The lockfile stayed the same; the evidence changed.
The product changes too. Adding team seats introduced a Principal, a Membership, tenant roles and new passwordless entry paths. Together they changed what an authenticated user means, how access should end and which database row is authoritative.
Deployment can invalidate a code-level conclusion. Database Row-Level Security policies defined in a migration do not prove that tenant isolation is active: a web process connecting as the table owner can bypass them. We test RLS with a restricted role in CI and check the effective database role in staging and production. The control is only effective when the deployed application is forced to use it.
A useful review records what was inspected, what was executed, what remains outside the evidence and what demands immediate action.
Security failures live at boundaries
A line-by-line review is useful, but security failures often live between parts of the system. An email address is profile data, a login identifier and a link to an owner identity. Changing it must update all three meanings, invalidate sessions on every device and burn outstanding credentials for the old identity. Updating one column is locally correct and systemically wrong.
Removing a team member has the same shape. Their existing sessions must stop working immediately, and authorization must use the current database membership rather than the role stored in an older cookie.
Deletion stretches past the relational cascade into object storage, billing-provider data and login identities that may no longer have a foreign key to the account. CI includes the machine, credentials and supply chain behind the test command. Public tokens bring decisions about lifetime, replay behaviour, HTTP method and the session they can create. Each creates a boundary where locally correct behaviour can still produce a system-level security failure.
Our review inventory follows those boundaries across authentication, tenant isolation, public tokens, billing, uploads, privacy, AI cost controls, CI/CD, dependencies and live configuration.
What AI can and cannot verify
We use AI heavily in this process. Our review bench includes the latest OpenAI and Anthropic models available to us. We will add independent models and review methods where their access and data-processing terms meet our requirements.
AI can trace an identity through routes, server actions, session validation, Prisma relations and tests. It can compare an implementation with the previous audit, inspect build artifacts, run the application’s checks and notice when documentation promises a purge job that the worker never schedules. That persistence is valuable across a large codebase.
It can also produce a plausible security story that is wrong: treating a hardening preference as a vulnerability, trusting a comment over the code or assigning severity without a credible path from attacker to impact. A repository alone cannot verify a live production setting.
The model does not sign off security. I do. Every actionable finding must show the trust boundary, concrete impact, source evidence, required outcome and regression test. I challenge findings that cannot do that and reject invented backlog work when the existing control is proportionate for a SaaS product at our stage.
Evidence from code, CI and production
To close the identity findings described here, “we tested it” meant a TypeScript check, more than 1,300 automated tests, a production build and a fresh dependency audit. We also applied every migration to a clean database, connected as the restricted application database role and ran cross-tenant probes in both directions.
For deployed state, we confirmed that staging and production use the restricted application database role, compared secret names and digests without retrieving values, and inspected the security headers returned by the production login page.
These checks answer different questions. A green test suite does not overrule a dependency advisory, and a clean dependency audit does not verify tenant isolation. Each control needs its own evidence.
We do not run destructive experiments against production. The report names what was not verified, including provider IAM settings, storage encryption policy and a full authenticated penetration test. Unknown is a valid result.
From findings to permanent controls
Findings are closed only when the fix is encoded in the product and protected by regression tests. Here, account-email changes invalidate older sessions and migrate the owner identity, while removing a team member revokes existing access.
We verified tenant-isolation behaviour locally and in CI, and confirmed that staging and production use the restricted database role required to enforce it. In the scenarios tested, we found no evidence of cross-tenant data access.
We do not publish open exploit paths or secret material. Detailed reports stay in our engineering workflow, and reviews run after material identity, provider and deployment changes, as well as on a regular schedule.
A clean report is comforting. A process that can interrupt the roadmap when the evidence demands it is more useful.