An unscoped GitHub token and an unprotected branch produce the same SOC 2 finding

The GitHub side has one failure mode that decides how much the rest is worth. Shashikumar reports that the branch-protection endpoint silently returns "not found" when a token lacks sufficient scope, and that from the API response alone this cannot be told apart from a genuinely unprotected branch ^[8]. Those two states mean opposite things on an evidence page. One says a repository does not require a review before merge to its default branch, which is the CC8.1 check ^[4]. The other says the collector could not see. The post describes the edge case and does not say how the collector separates the two.

The same class of problem sits one level up. "GitHub's API treats personal accounts and Organizations as different resource types with different endpoints", Shashikumar wrote ^[7]. His validation run used a personal account with 14 repositories ^[9]. A team with an Organization is calling endpoints that run never touched.

Count the control IDs and the collector touches four criteria: CC6.1 from both collectors, plus CC6.6, CC7.2 and CC8.1 ^[17]. The collector runs five check groups across two providers ^[3][4]. Change management is the thinnest of the four. The manual labour the tool is aimed at includes documenting change-management approvals by hand ^[13]. What the GitHub collector reports is whether the default branch requires a pull request review ^[4], a configuration setting.

The post prices each check at five minutes in isolation and says that a dozen-plus IAM users and repositories turns it into hours of repetitive work, redone every audit cycle ^[16]. Apply that five minutes per repository and the branch-protection sweep across his 14 repositories comes to about 70 minutes ^[18]. The collector makes one API call per repository. So the saving scales with how many items you own, and 14 coursework repositories sits at the small end.

On validation he is straight about what the run proves. He remediated two gaps, MFA on one IAM user and a branch-protection rule on one repository, then re-ran the collection to confirm the tool detected the fixes, and left every other finding unchanged so the output could not come back as a false all clear ^[11][12]. "These findings were real, not staged", he wrote of the initial results, most of them from older accounts used for certification study and coursework ^[10]. A compliance tool whose first target was the author's own study accounts has at least been pointed at something capable of failing. Two fail-to-pass transitions is a smoke test on the detection path ^[20].

Against Vanta, Drata and Secureframe, which he credits with hundreds of integrations and continuous cross-framework monitoring ^[14], Shashikumar calls his own tool "a scoped, free starting point for a small team's first audit cycle" ^[15]. I would use it for exactly that and not present its output as monitoring. The checks it automates are the ones a small team now does by screenshotting IAM console pages and exporting access logs ^[13], and the dashboard fires them on demand ^[6].