WeSCE: A Benchmark for Measuring Security Drift in LLM-Driven Code Editing

Details

AI coding assistants are routinely evaluated on correctness and style, but rarely on whether they introduce or worsen security vulnerabilities. WeSCE fills this gap with 400 executable programs drawn from real-world code, covering four editing task types: feature addition, feature removal, bug fixing, and refactoring. The benchmark operates under what the authors call weak-security constraints — tasks specify only functional objectives, mirroring actual developer use of tools like Copilot or Cursor. To measure security drift, WeSCE proposes a continuous risk representation rather than a binary pass/fail. It aggregates heterogeneous vulnerability signals into a single score, then defines three drift measures: changes in overall risk, worst-case severity, and vulnerability distribution shift across code transformations. This multi-scale view separates average-case behavior from tail risks. The work is primarily a benchmark contribution rather than a model evaluation, so it does not yet report which LLMs fare best or worst. Its main value is establishing a reproducible, quantitative methodology for security evaluation — currently absent from mainstream coding-assistant assessments. Practitioners building or evaluating AI code-editing pipelines now have a concrete tool to surface latent security regressions.