GPT-6 Astra Achieves 87.4% on MultiLangBench for COBOL and Fortran Coding

Seed story: "Advancing coding in low‑resource programming languages" (OpenAI) · search original Written from facts verified across 3 report(s) — original explainer, not a copy or translation. Sources at the end.

OpenAI’s latest research update reveals a dramatic leap in AI coding agents' ability to handle legacy and low-resource languages, with GPT-6 Astra achieving an 87.4% pass rate on the new MultiLangBench evaluation. This is critical for developers maintaining critical infrastructure, as COBOL still powers over 40% of online banking systems and Fortran remains essential for nuclear safety analysis. The findings also highlight a significant cost-performance shift, where newer models like GPT-6 Luna deliver competitive results at a fraction of the API cost of previous generations.

OpenAI Releases MultiLangBench and GPT-6 Coding Metrics

OpenAI’s October 19, 2026 research update introduces MultiLangBench, a new evaluation designed to test coding proficiency across 20 languages. The benchmark requires models to build a SQLite-compatible database engine from scratch, providing a rigorous measure of structural understanding rather than simple syntax generation.

Key performance metrics from the update include:

GPT-6 Astra achieved an 87.4% average test pass rate.
GPT-6 Luna recorded a 40.3% pass rate.
GPT-5.6 Sol, the previous generation, managed only 26.8%.

These results highlight a significant leap in handling low-resource languages. For developers, this suggests that modern models can now reliably scaffold complex database logic in legacy environments, reducing the manual effort required to validate code correctness across diverse technical stacks.

Why Legacy Languages Dominate Critical Infrastructure

The stakes for legacy code are rarely hypothetical. COBOL remains the backbone of critical financial infrastructure, supporting more than 40% of online banking systems, 80% of in-person credit card transactions, and 95% of ATM transactions. A single syntax error or logic flaw in these systems can halt global commerce.

Similarly, Fortran continues to underpin high-stakes scientific and industrial operations. It is utilized in weather forecasting models and nuclear reactor safety analysis, where computational accuracy is a matter of public safety.

For developers, this means that AI coding assistants are no longer just tools for modern web apps. They are becoming essential partners for maintaining and migrating the aging codebases that keep the physical and financial world running.

Analyzing the 87.4% Pass Rate on MultiLangBench

The 87.4% average pass rate for GPT-6 Astra on MultiLangBench marks a decisive leap in legacy code comprehension. This metric, which measures the ability to build a SQLite-compatible database engine from scratch across 20 languages, highlights a dramatic improvement over previous generations.

GPT-6 Astra: 87.4% pass rate
GPT-5.6 Sol: 26.8% pass rate
GPT-6 Luna: 40.3% pass rate

This threefold increase over GPT-5.6 Sol suggests that modern models now possess robust structural understanding of niche syntax. For developers, this means AI assistants can reliably parse and generate code in languages like COBOL and Fortran without extensive manual correction. The capability reduces the cognitive load required to maintain critical infrastructure, allowing teams to integrate legacy systems into modern pipelines with greater confidence and fewer integration errors.

Cost Efficiency in COBOL Migration Workflows

For teams managing legacy modernization, the economic impact of these advancements is just as critical as the technical gains. GPT-6.1 Sol demonstrates a compelling value proposition by achieving 86.6% accuracy in COBOL migration tasks at low reasoning effort. This performance far exceeds the 5.4% accuracy recorded by GPT-5.6 Sol, while simultaneously reducing operational costs by 39%.

This cost efficiency allows organizations to scale migration projects without proportional budget increases. Key advantages include:

Significant reduction in per-token processing fees.
Higher accuracy rates compared to previous generations.
Lower barrier to entry for large-scale codebase audits.

Developers can now integrate these models into automated pipelines, transforming what was previously a manual, error-prone process into a cost-effective, high-accuracy workflow.

Fine-Tuning vs. In-Context Learning for Low-Resource Languages

A 2025 study by Alessandro Giagnorio, Alberto Martin-Lopez, and Gabriele Bavota clarifies the optimal approach for handling low-resource languages. Their findings indicate that fine-tuning is typically the superior strategy for smaller LLMs, where the model lacks sufficient inherent knowledge. This method allows developers to inject specific syntactic rules for languages like COBOL or Fortran directly into the model’s weights.

Conversely, the research highlights a distinct shift for larger models. As LLM size increases, in-context learning becomes more effective, leveraging the model’s broader general capabilities without additional training. Notably, very large LLMs may actually deteriorate in performance when fine-tuned, a phenomenon attributed to insufficient training data for these niche domains.

For engineering teams, this distinction dictates tooling choices:

Use fine-tuning for cost-efficient, smaller models.
Rely on in-context learning for frontier, large-scale models.
Avoid fine-tuning massive models to prevent performance degradation.

Integrating Legacy Code Support into Your Dev Pipeline

Modernizing legacy systems requires a strategic blend of model selection and workflow design. For teams migrating critical infrastructure, such as the banking systems where COBOL still dominates, GPT-6.1 Sol offers a compelling balance of accuracy and efficiency. At low reasoning effort, it achieves 86.6% accuracy on migration tasks, a significant leap from previous generations, while costing 39% less than GPT-5.6 Sol. If your pipeline demands higher precision, increasing the reasoning effort pushes this figure to 97.9%, ensuring robust refactoring for complex codebases.

To optimize your development workflow, consider the following approach:

Tiered Model Usage: Deploy GPT-6 Luna for initial code analysis and documentation, leveraging its 78% lower API cost compared to GPT-5.6 Sol.
Targeted Refactoring: Reserve GPT-6.1 Sol for high-stakes migration tasks where the 86.6% baseline accuracy reduces manual review time.
Context Management: Rely on in-context learning for larger models, as research indicates fine-tuning can degrade performance in very large LLMs due to insufficient low-resource training data.

By aligning model capabilities with specific pipeline stages, developers can streamline legacy modernization without inflating operational costs.

FAQ

What is MultiLangBench and how does GPT-6 Astra perform on it?

MultiLangBench is an evaluation developed by OpenAI that tests a model's ability to build a SQLite-compatible database engine from scratch across 20 different programming languages. GPT-6 Astra achieved an average test pass rate of 87.4% on this benchmark, significantly outperforming GPT-5.6 Sol, which scored 26.8%.

How effective is GPT-6.1 Sol for COBOL migration tasks?

In a COBOL migration evaluation, GPT-6.1 Sol achieved 86.6% accuracy at low reasoning effort, compared to just 5.4% for GPT-5.6 Sol, while costing 39% less. At higher reasoning effort, GPT-6.1 Sol reached 97.9% accuracy on the same evaluation.

Why is improving coding support for languages like COBOL and Fortran important?

COBOL supports more than 40% of online banking systems and 95% of ATM transactions, while Fortran is utilized in critical areas like weather forecasting and nuclear reactor safety analysis. These languages are considered low-resource, making advancements in AI coding capabilities essential for maintaining legacy infrastructure.