JFinBench
Intro
JFinBench is the Japanese finance benchmark Sakana AI published. The official blog post is dated 17 November 2026 and titled “JFinBench: 最新の金融資格試験を用いた日本語金融ベンチマークの公開.” The official @SakanaAILabs account had already announced the same work at 08:53 UTC on 16 November. This scan files that official name, which had not been added in earlier slots.
According to Sakana, JFinBench is built from the 2026 editions of six Japanese finance certification exams. The official long post names the CPA exam, the tax accountant exam, and FP Grade 1 among them, for 321 questions in total, including not only multiple-choice items but also tax calculations, written answers, and questions that require reading figures. The company stresses that tax and accounting rules change often, so a finance benchmark needs to be refreshed from the latest exams.
Public resources listed by Sakana include the paper “JFinBench: A Continuously Refreshable Japanese Financial Benchmark from the Latest Certification Exams,” the SakanaAI/JFinBench dataset and code, and a leaderboard site. It should not be described as a US finance benchmark, or conflated with earlier Japanese finance tests that covered only a narrower set of multiple-choice questions.
Advertisement
Explain
Sakana presents finance as a domain where benchmark building still lags areas such as coding: an open-weight model that looks close to the frontier on major or English-language benchmarks can show a much larger gap once Japanese institutional knowledge is required. The blog gives the example that Kimi K3.1 Pro, which has been reported as comparable to GPT-6 Astra on major benchmarks, still trails GPT-6 Astra by about 10 points on JFinBench. Both the summary and the official long post say the main cause of failure is not calculation, but missing knowledge of Japanese institutions; when models can use a terminal and the web, scores rise as they look up statutes, but the gap remains.
The other official claim is the construction method. Sakana proposes a human-in-the-loop agentic pipeline in which a coding agent converts exam PDFs into scorable items and humans verify every question. The official post says visual verification takes about 12 hours, so the benchmark can be updated with relatively little labor when new exams appear. That matches the paper’s “Continuously Refreshable” subtitle.
Exact per-model scores, tool versus no-tool comparisons, and cost per question should be taken from the official figures and leaderboard. This entry only repeats the roughly 10-point gap and the stated cause of failure from the summary, and does not add precise leaderboard numbers that were not re-checked here.
Usage
The first-party entry point is the blog sakana.ai/jfinbench; the official announcement is the @SakanaAILabs post. The paper, the SakanaAI/JFinBench dataset and code, and the leaderboard are the accompanying resources Sakana lists.
It is meant for measuring how models handle Japanese tax, accounting, and related certification tasks — especially whether their knowledge matches 2026 rules — rather than generic math or English-language finance questions. Sakana also notes that scores change when tools are allowed and models can look up statutes, so comparisons should keep the tool and no-tool settings separate.
When citing results, go back to the official figures and leaderboard, and name JFinBench rather than a generic “Japanese finance test.” This entry is a catch-up filing of an official name published on 16–17 November that earlier slots had not added.