You Pushed AWS Keys to a Public Repo
Scenario
A developer commits a config file containing a live AWS access key and secret, pushes it to a public GitHub repository, and realizes the mistake five minutes later. The key has permissions on S3 buckets and compute resources, and the repository already has forks. The clock is very much running.
The Quick Fix on the Table
The teammate's instinct: delete the file, rewrite the commit out of history, git push --force, done. The keys are no longer visible in the repo, so the leak is "cleaned up".
Interview · Round 1
The quick fix is on the table and the room is waiting for your call. Would you sign off on it? Take a position and justify it — out loud or on paper — before revealing the analysis.
Why the Quick Fix Fails
- The secret was harvested before you finished typing the force-push. Automated scanners watch public commit streams continuously and typically try leaked cloud keys within a minute or two of the push. Five minutes of exposure means you must assume the key is in hostile hands already.
- Force-push cleans one copy of many. The commit lives on in forks, in clones, in the platform's caches and event feeds, and in every scanner's database. Rewriting your repo's history changes what future visitors see; it retrieves nothing.
- It treats an incident as a tidiness problem. The dangerous object is the live credential, not the file. As long as the key works, an attacker can spin up compute, exfiltrate S3 data and create their own persistence — regardless of what your git history says.
- It skips the question that matters: what already happened? Cleaning history without auditing usage means any attacker-created users, keys or resources from those five minutes stay in the account, quietly.
- It teaches the wrong reflex. If the team's playbook for leaked secrets is "make it look like it never happened", the next leak — maybe of a credential that can't be seen in a public repo — gets the same non-response.
Interview · Round 2
The interviewer nods: “Fine, the quick fix is off the table. So what exactly would you do — step by step?” Sketch your plan before revealing the approach.
The Right Approach
- Kill the credential first — before any git surgery.
aws iam update-access-key --access-key-id AKIA... --status Inactive(then delete it) the moment the leak is known. Every other step can wait; this one is measured in seconds of exposure. - Restrict the blast radius while you investigate. If the key belonged to a user rather than a role, attach a deny-all policy to that user temporarily, in case other credentials or sessions exist for it.
- Audit what the key did. Search CloudTrail for the key ID over the exposure window and beyond: unexpected regions,
RunInstances, IAM calls creating users/keys/roles, S3 reads. Check billing anomalies too — cryptomining shows up there first. - Hunt for persistence. Attackers use stolen keys to mint durable access: list IAM users, access keys, roles and trust-policy changes made in the window, and remove anything you didn't create. A compromised account is not clean just because the original key is dead.
- Rotate everything the file touched. If that config held other secrets — database passwords, API tokens — treat them all as burned and rotate them at their sources, updating consumers as you go.
- Now clean the history, as hygiene rather than remediation. Remove the file with
git filter-repo(or the BFG), force-push, and contact the platform about cached views — understanding that this is tidying a dead secret, not containing a live one. - Make recurrence hard. Add pre-commit secret scanning (
gitleaks,git-secrets), enable the platform's push protection, and move the application to IAM roles or a secrets manager so there is no long-lived key to paste into a file next time.
Interview · Round 3
Final pushback: “Your plan costs more time and money than the quick fix. Convince me.” How do you defend your position under pressure?
How to Defend It
- "Public-repo scanners try leaked AWS keys within minutes. Whatever we do to git history, we have to assume someone already has the key — so revocation is step one, not cleanup."
- "Force-push edits our copy of history. The forks, the clones, and every scanner that already saw the commit keep theirs. You can't unpublish a secret; you can only kill it."
- "The incident isn't 'a file is visible' — it's 'a credential with S3 and compute access was live in public for five minutes'. CloudTrail tells us whether those five minutes were quiet or not."
- "Stolen keys get used to create fresh keys. If we don't sweep IAM for persistence, we could rotate the leaked key and still be compromised tomorrow."
- "History cleanup is fine — as the last step. And the lasting fix is push-protection plus no long-lived keys in files at all, so this class of incident can't repeat."