DAYJOB: Tax: How Far Are AI Agents From Real Tax Work?

I. Background/Context

Taxation has long been a field highly dependent on human professional judgment. From individual tax filings to corporate mergers and acquisitions, from cross-border transactions to estate trusts, each case involves a large amount of unstructured documents, complex regulatory applications, and key information that the client may not explicitly state but will affect the outcome. In October 2026, Surge AI released a benchmark called DAYJOB: Tax, designed to evaluate the performance of AI agents in real-world tax tasks. It is not a series of standardized multiple-choice questions; instead, it places the agent directly into end-to-end tasks designed by tax experts to see how well it can perform from reading documents to forming conclusions.

This benchmark has sparked discussion on Hacker News, with the focus not on which model ranks first, but on whether AI agents have the understanding, judgment, and responsibility required to handle professional tax work. Surge AI labeled DAYJOB: Tax as the “AI Agent Professional Tax Task Benchmark.” Only 2 tasks are available as samples publicly, and the full version can only be obtained by applying to the official repository. Its significance lies more in establishing a standard for measuring agents’ real-world capabilities in professional services than in publishing a downloadable dataset.

Some discussions point out that the ultimate bearer of tax responsibilities is always the taxpayer. Even if a accountant makes a mistake, the taxpayer cannot be exempted from tax obligations and can only pursue compensation afterwards. This means that any AI tool entering the tax field must not confuse “auxiliary advice” with “ultimate responsibility.” One of the values of DAYJOB: Tax is to present this responsibility boundary in a quantifiable way.

II. Why Now

There are two backgrounds driving the creation of this benchmark. First, the capabilities of AI agents are expanding from general dialogue to vertical professional tasks. Over the past year, agents’ abilities in coding, research, and law have rapidly improved, and tax, as a field with dense documents, complex rules, and extremely low tolerance for errors, has naturally become the next target for evaluation. Second, tax work itself is undergoing digitization. The processing workflow involving PDFs, Excel files, emails, and scanned documents naturally aligns with AI agents’ ability to read multimodal documents. The emergence of DAYJOB: Tax occurs at the intersection of model capabilities and industry needs, providing a quantifiable method for evaluating whether agents can fulfill the role of “professional assistants.”

However, this also raises a key question: What standards should we use to evaluate AI agents when they enter high-risk fields like taxation? DAYJOB: Tax’s answer is “pass@1, 95% pass rate.” This means that the agent must not only calculate numbers correctly but also find all evidence, identify key issues, and deliver qualified results. This standard is much more stringent than general question-answering tasks because it simulates a scenario where the client hands a real problem to the agent, and the agent must provide an answer right from the first attempt.

III. Core Mechanism

The core design of DAYJOB: Tax simulates the workflow of a real tax agent. Each task includes a set of documents from real scenarios, such as PDFs, Excel files, Word documents, emails, images, CSVs, JSONs, etc. The agent must extract information, perform calculations, identify risks, and provide final conclusions. The tasks cover eight tax practice areas: personal and household taxation, small business taxation, partnerships, property and investments, estate gifts and trusts, international taxation, corporate transactions, and tax audits and disputes.

评分采用 rubric 方式，从计算、证据、关键问题、最终交付物等维度打分，需要达到 95% 的阈值。这意味着代理不能只给出看似合理的答案，而必须在细节上也经得起核对。任务由真实税务专家设计，并经过正确性、完整性和可解性三层审核，确保任务既有现实代表性，又有明确的评判标准。

官方示例中的两个案例很能说明问题。一个是 Anna 卖房案：代理正确判断了两个交割日期都满足自住免税条件，却遗漏了 2027 年 8 月后可能丧失资格的截止期限。另一个是分居夫妇退休账户案：代理没有核实婚姻状态，而不同州对普通法婚姻的认定存在差异，这可能导致税务后果出错。这两个例子共同说明，DAYJOB: Tax 考察的不仅是计算正确性，还有对客户未明言信息的追问能力。

与传统税务基准相比，TaxBench 更偏向高难度税务问答，TaxCalcBench 侧重报税准确性，TaxLab 则模拟代理与纳税人对话并填写联邦个税表。DAYJOB: Tax 的不同之处在于，它更强调端到端的专业判断：发现客户没有提到但会影响结果的关键问题，处理互相冲突的证据，并在大量文件中找到支撑结论的依据。

IV. Numbers and Boundaries

According to official data, DAYJOB: Tax includes 50 tax task environments, totaling 1,789 files, with a median of 30 files per task, and the most tasks containing 123 files. PDF accounts for the majority of files, with 1,487 copies, followed by 117 Excel files, 38 Word files, as well as emails, images, CSV, JSON, text files, etc.

The official rankings show that even the strongest models only have a pass rate of around 10%. Claude Sonnet 5.5 ranks first with 10.8%, Claude Fable 5.1 with 9.6%, Claude Opus 5.5 with 9.2%, Muse Wonder 1.0 with 6%, Grok 4.7 with 5%, GPT-6 Astra with 4.8%, GPT-6.1 Sol with 4%, Gemini 4 Argon with 3.2%, and Kimi K3 with 0%.

These numbers indicate that current AI agents still have a significant gap from reliably completing professional tax work. However, it should be noted that there are only 2 tasks in the publicly available samples, and 48 complete tasks need to be applied for at benchmarks@surgehq.ai, so they cannot be called “open-source datasets” or “complete public availability.” Additionally, the ranking data comes from Surge AI’s official release, and no independent third-party re-measurement has been seen yet, so caution should be exercised when citing these numbers.

From discussions on Hacker News, a core question is: Is this benchmark testing the model’s inherent capabilities, or the prompt engineering and tool design? If the agent lacks scaffolding for task types, can it organize itself in context and track all requirements? There is no clear answer to this question yet, but it reminds us that when citing pass rates, we cannot simply equate numbers with the model’s ability.

V. How to Use

For tax professionals, DAYJOB: Tax provides a reference framework for understanding the limitations of AI agents. When selecting or evaluating tax AI tools, consider their evaluation criteria: whether they have accurate calculation capabilities, whether they can find sufficient evidence, whether they can identify key issues not mentioned by the client, and whether the final deliverables are complete and compliant. If a tool performs unstable in these aspects, it should be classified as an auxiliary tool, not a substitute for human judgment.

For AI product teams, DAYJOB: Tax reveals the complexity of handling multiple documents, formats, and long chains that agents need to handle in tax scenarios. When designing products, one should not only pursue the accuracy of the final answer but also ensure that evidence chains, risk warnings, and key issue prompts are essential outputs, along with establishing a manual review mechanism. Products can focus on three areas: “draft generation,” “risk pre-checking,” and “document indexing,” rather than claiming to complete the entire tax process.

For enterprise users, DAYJOB: Tax’s performance reminds us not to overestimate the independent execution ability of current AI agents in professional tax work. Even the strongest models have an accuracy rate of only about 10%, indicating that in the foreseeable future, AI should be considered as “draft generators” and “risk pre-checkers,” with final judgment still needing to be completed by human professionals. When introducing tax AI, enterprises should prioritize its use in areas with high standardization and large tolerance for errors, while maintaining a complete review process.

VI. Value Judgment

The greatest value of DAYJOB: Tax lies in transforming the question “Can AI perform professional tax work?” from a subjective judgment into a quantifiable assessment. With real documents, real tasks, and strict evaluation, the model’s capabilities are fully revealed in professional scenarios. The highest accuracy rate of 10.8% also shows that current AI agents still have significant shortcomings in professional judgment, complex document understanding, and risk identification.

At the same time, this benchmark raises a deeper question: When AI agents enter high-risk professional service fields such as tax, law, and healthcare, is the “accuracy rate” sufficient? Tax results often come with legal responsibilities and economic consequences. Although a 95% threshold is strict, a 5% error in reality can still lead to serious consequences. Therefore, DAYJOB: Tax should be understood as a capability assessment tool, not a security certification tool. It tells us what the model can do, but it cannot tell us what the model can do safely.

From an industry perspective, DAYJOB: Tax also sets a capability baseline for AI applications in professional service industries. After it, any product claiming to replace tax experts must first explain how it performs on similar benchmarks and which aspects still require human intervention. This is much more important than simply promoting “AI can file taxes.”

Conclusion

DAYJOB: Tax is a capability assessment of AI agents in the professional tax field. With 50 real tasks, nearly 1,800 documents, and strict scoring criteria, it reveals the true level of the most advanced models in professional tax work: a pass rate of about 10% indicates that AI can handle some simple tasks, but there is still a significant gap from independently undertaking professional tax work. For the industry, this means that the application of AI in tax scenarios should continue to focus on “human-machine collaboration,” using AI as a tool to improve efficiency rather than as a substitute for professional judgment. In the future, as model capabilities improve and more third-party retests emerge, DAYJOB: Tax may become an important reference for measuring the maturity of tax AI.

Reference Sources

• Surge AI official blog: DAYJOB: Tax — Benchmarking AI Agents on Professional Tax Work
• Hugging Face public sample: surgeai/DAYJOB-tax (MIT licensed)
• Hacker News discussion: objectID 50275626