Developer benchmarks Claude Code plugins across four tech stacks
11/17/2026 — 11/17, 09:15·1 sources (all)·1 reports (all)
Key takeaways
- A developer built Claude Code plugins — skills, subagents and guard hooks — for four tech stacks and compared runs with and without them on…
- the other three stacks passed all six.
Background & analysis
On 2026-11-17 at 09:06, the AI section of DEV Community published a hands-on evaluation testing how much installing stack-specific plugins changes what Claude Code actually does on real repositories. The author wrote one Claude Code plugin for each of four stacks, with each plugin bundling three components: skills, subagents, and a guard hook.
The test ran against pinned commits of four public repositories. For every stack, the author used the same six prompts and ran each one both with the plugin installed and without it, creating a controlled with-without comparison. Every case was executed three times and then scored by a checklist plus a review model. A case counted as passing when its score reached 0.75 or higher.
The results showed a split. Drupal passed 5 of the 6 cases, while the other three stacks cleared all 6 cases with the plugin in place. The entire pipeline relied on Claude Code's built-in plugin eval runner and its --ablation switch. The plugin eval runner executes each case inside a scratch copy of the repository, running it once with the plugin and once without it for the ablation, then produces the scores that feed the pass threshold.
So far, only this single round of with-without results has been made public. The article does not mention any follow-up retests, wider stack coverage, or plans to extend the setup, leaving the comparison at this one published measurement.
AI-generated from 1 reports · updated 3 hours ago
Related AI tools
- Heat index
- 92
- All sources
- 1
- All reports
- 1
- First seen
- 3 hours ago
Reports on this story headlines open the original
The author wrote a Claude Code plugin for each of four stacks — bundling skills, subagents and a guard hook — then ran the same six prompts with and without the plugin against pinned commits of four public repos, three runs per case, scored by file checks and a judge model. Drupal passed 5 of 6 cases; the other three stacks passed all six, with 0.75 as the passing threshold. Everything ran through Claude Code's built-in plugin eval runner with its --ablation flag.
DEV Community · AIAI score 53
Other stories people are talking about
- Heat index 90NewDeveloper builds a five-step post-generation check pipeline around Claude Code1 sources
- Heat index 46Developers Use PreToolUse Hooks to Stop Claude Code from Overwriting .env1 sources
- Heat index 111Claude Code Code Mode can be enabled via an environment variable2 sources
- Heat index 41Goldman Sachs revokes trader's Claude Code access, sparking workflow debate1 sources
- Heat index 85NewNadella says Claude Code caught Microsoft off guard as IDEs fade1 sources
How is heat calculated?About the methodHide
Heat counts how many independent sources covered a story in the last 48 hours: one source counts once no matter how many posts it published, decaying with a 24-hour half-life. What ranks first is what many people are talking about.
This page aggregates public feeds. Headlines and summaries are machine-organized and remain the property of the original authors; verify important facts at the source. If you believe a headline or summary infringes your rights, email the contact address in the footer with the page URL and basis of your claim, and we will remove or replace it after review.
- Surge
- Discussion rising fast
- New
- First report within 6 hours
- Rising
- Still gathering discussion

