Tencent Unveils WorkBuddy Bench: A Coding Intelligent Agent Testing Ground Integrated with Code, Web, Office, and Security

Historically, coding agents have been restricted to narrow domains—such as bug fixing on SWE-bench or frontend tasks on Design2Code—while production benchmarks remain largely inaccessible to external review. Tencent addresses this gap with WorkBuddy Bench, a multi-domain evaluation suite detailed in a recent arXiv paper. Its defining characteristic is not task volume, but a design architecture explicitly engineered to prevent answer memorization.

This benchmark covers four professional domains: software engineering (repository-level code), frontend development (web artifacts), business operations (multi-file office workflows), and cybersecurity (red and blue team scenarios). The suite comprises 80, 70, 50, and 60 tasks respectively, totaling 260. Crucially, no tasks originate from public question banks; instead, they are reverse-engineered from real code commits, pull requests, or business contexts, then rephrased into concise, colloquial, role-playing prompts. This approach ensures that online searches for corresponding PRs or commit clues yield no results, effectively blocking memorization. Since the dataset is fully open—including directories, environment images, evaluation tools, test cases, and reference solutions—its contamination resistance relies on this construction method and version control rather than obscurity.

While all four subsets share a unified directory structure, each employs distinct validation metrics, making direct score comparisons between subsets invalid. Consequently, Tencent does not publish an aggregate suite score. Code tasks are evaluated via hidden test pass rates; web tasks combine rule checks with LLM/VLM and agent evaluations, requiring artifact delivery along a specified path without live internet access; office tasks use deterministic rule checks weighted 0.70–0.95 alongside evidence-based LLM evaluation; and security tasks rely on deterministic scoring.py scripts run three times for averaging, excluding large models as judges. The security subset implements five anti-cheating layers: disabling literal scanning, renaming inputs, masking tampered tests, encoding dependencies, and using low-weight bait tasks. It includes 38 red team and 22 blue team tasks, with white-box audits anchored to real-world CVEs in binutils, curl, and nginx.

Task creation follows a standardized pipeline: source collection, rewriting into authentic requests, workspace assembly, isolation of assessment assets post-run, and packaging into independent directories. User data remains untouched throughout. Code tasks assign five distinct roles (developer, algorithm engineer, product manager, QA, operations), while security tasks assign professional personas. Deliberately incomplete information is provided—target files, patterns, and boundaries are withheld—forcing agents to retrieve context independently. Scoring assets are revealed only after execution, ensuring agents never encounter them during operation.

Evaluations occur in isolated containers with separated model and sandbox environments. The framework utilizes CodeBuddy Code (default) and Claude Code, employing increased reasoning efforts and a 200k context window, while disabling WebSearch and AskUserQuestion. This restriction is critical: disabling online search verifies whether contamination resistance mechanisms function as intended.

The leaderboard ranks multiple model families (scores 0–100, thinking mode, three averages). Claude Opus4.8 dominates the top positions across code, web, office, and security, securing five first-place rankings; GLM-5.2 leads two security categories, and GPT-5.5 tops office cc. The framework demonstrates high sensitivity, particularly in security, where average absolute rank changes reach 8.6 points. Under the cc framework, Claude Opus4.8 rejected 13 answers, compared to GPT-5.5’s two rejections under cbc. Regarding efficiency, GPT-5.5 generates the fewest tokens while maintaining competitive scores, whereas DeepSeek-V4-Pro executed 44 code rounds with high token throughput, reflecting a more resource-intensive approach.