top of page

Mathebula ESG AI Review Flags Data Gaps

48 minutes ago
11 min read
Illustrative photograph. Photo: RDNE Stock project / Pexels.

Miehleketo Mathebula and colleagues reviewed 70 studies and identified a persistent constraint on financial AI: the information underneath increasingly sophisticated models remains fragmented. Published November 21, 2026, the Mathebula ESG AI review examines environmental, social, and governance analysis, which assesses companies’ sustainability and governance characteristics. Its central argument challenges a familiar development priority. Better models alone cannot establish reliable results when disclosures, definitions, and evaluation datasets do not line up.

The tension concerns model improvements versus the data infrastructure needed to make those improvements useful across markets. Researchers report progress on individual tasks, including extracting sustainability indicators and analyzing corporate disclosures. Yet performance within a selected dataset does not establish reliability under different reporting conditions. Historical research comparing providers including MSCI and Sustainalytics offers an important reference: even established ESG ratings disagree.

That puts pressure on developers selling financial analysis systems and buyers evaluating their claims. The review does not establish that every ESG application fails. It identifies gaps between published experiments and dependable operation, particularly across jurisdictions. Its proposed architecture also remains conceptual, with implementation and performance testing still ahead.

What the Mathebula ESG AI Review Actually Examined

The new contribution is a structured assessment of research limitations, rather than a newly tested financial AI system.

Mathebula, Abiodun Modupe, Vukosi Marivate, and Elda Du Toit published their ESG AI review in Discover Artificial Intelligence. They examined peer-reviewed journal studies published from 2017 through 2025. Their search took place on January 7, 2026, across IEEE Xplore, ScienceDirect, SpringerLink, and MDPI.

The initial search identified 1,623 records. After removing 57 duplicates and making 78 preliminary exclusions, the researchers screened 1,488 titles and abstracts. They sought 131 full reports, but 13 were unavailable. Full-text assessment covered 118 reports, with 48 excluded and 70 retained.

Those counts describe the review’s selection process. They do not measure how frequently deployed products fail, or how accurately an AI system predicts financial outcomes. Keeping those categories separate matters because literature summaries can acquire stronger meanings as they move through headlines and commercial presentations.

The researchers organized the evidence around modeling methods, data characteristics, analytical tasks, and methodological constraints. They considered applications such as extracting indicators, assessing climate-related risk, evaluating disclosures, and identifying potentially misleading sustainability claims. These applications share information sources, but they do not share identical evaluation requirements.

For example, accurately locating a reported emissions figure is different from determining whether a company’s environmental claims misrepresent its actual behavior. The second judgment requires evidence beyond the wording of a disclosure. A model’s success on extraction therefore cannot establish its competence across every downstream task.

The review also distinguishes advances in individual methods from readiness across an entire workflow. A study can make a useful contribution while leaving document processing, reporting compatibility, or oversight unresolved. That distinction allows technical progress and operational limitations to coexist without treating either as evidence against the other.

South Africa serves as an illustrative emerging-market setting throughout the paper. The researchers did not geographically restrict their search to South African studies. Their broader argument concerns transferability: whether methods developed under particular disclosure conditions remain useful when those conditions change.

The publication date also needs context. Although the paper appeared in November 2026, its included studies stop at 2025. Readers should treat it as a synthesis of that evidence window. It does not provide a comprehensive evaluation of every model or commercial release introduced during 2026.

This scope makes the paper useful for assessing development priorities. It provides less support for sweeping claims about the current performance of particular products. The authors’ strongest contribution lies in connecting recurring research constraints to the requirements of future systems.

Financial AI Buyers Inherit the Disclosure Problem

Buyers cannot assess an ESG system’s reliability from its model name or headline accuracy alone.

The review’s findings place enterprise evaluation under pressure because the analytical output depends on decisions made before modeling begins. Corporate reports arrive with different structures, terminology, and levels of detail. A system must interpret those differences before it can make meaningful comparisons.

ESG data infrastructure includes the collection, organization, definitions, and controls supporting that interpretation. It determines which documents enter the system, how their contents become usable records, and whether their meanings remain consistent. Weaknesses at this level can travel into an apparently polished final answer.

Consider the difference between finding a governance statement and deciding what that statement establishes. A report can discuss board accountability without providing the evidence needed to assess its effectiveness. Identifying the passage correctly does not settle the substantive judgment.

This creates a practical distinction between assistance and assessment. A tool that helps an analyst locate relevant passages faces a different test from one generating company rankings. The latter must justify the relationship between source material, chosen indicators, and the resulting comparison.

The review identifies limited operational validation as a recurring constraint. Operational validation means testing whether a system works under its intended conditions of use. A selected research dataset offers evidence about that dataset. It does not automatically establish reliability across unfamiliar companies, document formats, or jurisdictions.

For developers, that shifts attention toward the boundaries of a claim. An extraction result should identify the indicators and reporting conditions tested. A scoring result should identify the target it reproduces. Broader claims need broader evidence, especially when users will compare companies subject to different disclosure requirements.

For enterprise buyers, the immediate implication concerns procurement evidence. A demonstration using familiar documents can show useful capabilities without resolving transferability. Evaluation becomes more informative when developers disclose missing inputs, ambiguous classifications, and the conditions under which human review remains necessary.

The same concern reaches analysts and knowledge workers using AI to accelerate research. A concise answer can hide disagreements between source documents. Preserving those disagreements becomes essential when the task depends on interpreting evidence rather than simply collecting text.

A searchable knowledge base provides a related information-management perspective: source material must remain accessible for inspection. That connection does not establish any product’s suitability for regulated financial analysis. It explains why retrieval convenience and analytical validity require separate evaluation.

The pressure identified here is therefore broader than a request for better model performance. Buyers need evidence about the surrounding process. Developers need to explain which parts of that process they have tested, and which assumptions still depend on the reporting environment.

Better Models Cannot Reconcile Undefined Comparisons

A system can reproduce an ESG rating accurately while leaving the meaning of that rating unresolved.

The central conflict in the Mathebula ESG AI review concerns the limits of pursuing model sophistication without resolving data comparability. A large language model, an AI system trained to process and generate language, can interpret varied wording. That capability does not establish that differently defined indicators measure the same thing.

The paper identifies incompatible schemas, which are structures used to organize data fields and their relationships. It also examines inconsistent definitions and dependence on proprietary datasets. These problems affect what a model learns and what an evaluator treats as a correct answer.

Commercial ratings illustrate the difficulty. MSCI, Sustainalytics, and other providers offer distinct approaches to assessing ESG characteristics. The historical comparison here concerns differences between methodologies. It does not establish which provider currently delivers superior results, or imply that disagreement necessarily reflects an error.

Research by Florian Berg, Julian Kölbel, and Roberto Rigobon examined ESG rating divergence across six prominent agencies. Published in 2022, their study attributed disagreement to differences in measurement, scope, and weighting. Measurement accounted for 56%, scope for 38%, and weighting for 6% of the divergence they analyzed.

That finding provides a useful historical reference for AI evaluation. A rating is a constructed assessment with methodological choices behind it. Agreement with a provider’s score therefore measures agreement with that provider’s approach. It does not independently establish a universal measure of corporate sustainability.

The distinction matters when proprietary scores become training targets or evaluation labels. A model might learn their patterns effectively while outside researchers cannot fully inspect the underlying methodology. Strong performance can then coexist with limited reproducibility and uncertainty about what the result represents.

ESG data infrastructure must address those interpretive choices alongside document processing. Matching field names is insufficient when their definitions differ. Combining records can make information easier to query while preserving incompatible assumptions underneath.

There is also evidence of institutional work toward better alignment. The IFRS Foundation issued sustainability disclosure requirements in June 2023. IFRS S1 addresses sustainability-related financial information, while IFRS S2 addresses climate-related disclosures. Their existence provides a reference for reporting, rather than proof of uniform implementation everywhere.

On May 2, 2024, the IFRS Foundation and EFRAG published reporting interoperability guidance. Interoperability means enabling different systems or frameworks to work together while preserving meaning. The guidance addresses alignment between international and European sustainability reporting standards and aims to reduce duplication.

These efforts complicate any claim that disclosure fragmentation is permanent or that no coordination exists. They also show why stronger language processing cannot replace institutional decisions about reporting definitions. AI systems operate within those decisions, including where standards remain distinct.

The paper’s argument therefore supports a change in emphasis rather than a verdict against model research. Better extraction and interpretation remain useful. Their value depends partly on whether developers can show that comparisons preserve the meaning of the underlying evidence.

The South African Example Shows Where Meaning Gets Lost

The paper’s document walkthrough illustrates an integration challenge, not a deployed customer result.

The researchers use a sustainability report from a Johannesburg Stock Exchange-listed firm to explain their proposed architecture. The example involves King IV, South Africa’s corporate governance framework. It gives the discussion a concrete setting without turning the paper into an empirical deployment study.

The report begins as disclosure material containing governance commentary, sustainability narratives, and financial information. Preprocessing, the preparation of raw content for analysis, separates and organizes those passages. That stage makes the document easier to analyze, but it does not resolve how its governance concepts correspond to other frameworks.

The difficult step concerns semantic alignment, which connects concepts according to their meaning. King IV concepts such as ethical leadership and stakeholder inclusivity do not necessarily map directly onto international ESG categories. A system must decide how to represent those relationships and where equivalence remains uncertain.

A model can retrieve a relevant paragraph without resolving that mapping. It can also generate a coherent explanation while relying on a debatable classification. In the authors’ walkthrough, uncertainty introduced during preparation or alignment can carry into later analysis.

That matters because the proposed downstream outputs extend beyond document search. They include governance-risk indicators, ESG scores, and flags for potentially misleading claims. Each output inherits assumptions from the steps preceding it.

The paper responds with a conceptual architecture linking five components. These cover preprocessing, semantic standardization, modeling, task-level analysis, and governance-facing outputs. The contribution is their organization into a connected design, with evidence requirements attached to the components.

The framework also emphasizes traceability, meaning the ability to follow an output back through its supporting evidence and processing decisions. In the King IV example, that includes the original passage, the mapping choice, and the modeling step. Those connections would allow reviewers to inspect where interpretation entered the result.

Retrieval, which locates information relevant to a query, can support that inspection. However, retrieving a passage does not establish that its interpretation is correct. A system needs evaluation of both the selected evidence and the reasoning connecting it to the output.

The authors explicitly state that their architecture has not been implemented, empirically validated, or performance tested. Its components describe design requirements for future investigation. The walkthrough therefore cannot support claims of improved accuracy, successful deployment, or demonstrated regulatory compliance.

This limit strengthens the paper’s usefulness when stated clearly. It identifies where future experiments should concentrate without presenting those experiments as completed. Readers can assess the proposed structure independently from evidence about whether it works.

The example also clarifies why cross-market AI validation requires more than adding documents from another country. Evaluators must examine the meanings and institutional assumptions attached to those documents. Geographic variety alone does not establish that a system preserves local governance concepts.

The Review Has Its Own Evidence Boundaries

A systematic review can identify recurring limitations without measuring their prevalence across the entire commercial market.

The Mathebula ESG AI review draws from English-language, peer-reviewed journal articles in four selected databases. Those choices create a defined evidence base and allow consistent appraisal. They also exclude research outside those sources and publication formats.

Conference papers, preprints, technical reports, and policy materials fall outside the included journal corpus. Some recent AI developments therefore remain beyond its scope. The authors acknowledge a tradeoff between consistent peer-reviewed evidence and coverage of a rapidly changing field.

Language restrictions also matter for an argument about international transferability. English-only inclusion can miss regional research and reporting contexts. A review concerned with uneven infrastructure must therefore recognize that its own available literature does not represent every market equally.

Publication bias adds another limitation. Studies reporting positive findings can receive more visibility than unsuccessful experiments. The researchers considered that risk, but they did not perform a formal publication-bias assessment using comparable effect sizes.

That detail limits the conclusions readers should draw from study counts. The frequency of a method in the literature is not its market share. The frequency of a reported limitation is not a measured failure rate across deployed systems.

The review’s appraisal process also needs careful interpretation. Two reviewers independently assessed selection and study quality. Their reported agreement supports consistency in those review decisions, rather than predictive accuracy for any ESG model.

A reasonable counterargument is that practical AI tools can improve without solving every reporting inconsistency. The reviewed literature includes task-specific advances. An analyst might gain useful assistance from narrower extraction or retrieval capabilities even when broader comparability remains unresolved.

That counterargument does not eliminate the review’s central concern. It changes the appropriate scope of a product claim. Evidence supporting a bounded task should remain tied to that task until testing supports a broader conclusion.

The distinction also prevents an opposite overstatement: missing published validation does not prove a particular commercial product performs poorly. Some evidence may remain private or fall outside the review’s criteria. Buyers still need access to relevant validation before treating broad claims as established.

The wider governance context supports attention to the full system. NIST’s voluntary AI risk framework, released January 26, 2023, addresses trustworthiness across design, development, use, and evaluation. It offers a governance reference rather than certification of any ESG product or proof of compliance.

Taken together, these boundaries keep the article’s conclusion proportionate. The evidence supports scrutiny of data conditions, evaluation practices, and deployment claims. It does not establish that model advances are irrelevant, all ESG ratings are useless, or every financial AI application shares the same weaknesses.

What Would Strengthen the Evidence Next

The next meaningful advance would connect improved models with transparent tests under difficult reporting conditions.

From November 2026 through February 2027, three developments would provide useful evidence about the review’s diagnosis. These are signals to monitor, rather than scheduled releases or predictions. Each would address a specific gap between isolated research results and dependable analysis.

The first signal is a benchmark release covering different jurisdictions, reporting frameworks, and languages. A benchmark is a shared evaluation dataset with defined tasks and scoring rules. Its usefulness would depend on transparent annotations and documentation of ambiguous cases.

For cross-market AI validation, the important question would concern performance differences across those conditions. Results that remain consistent on unfamiliar reporting environments would strengthen claims of transferability. Large unexplained gaps would reinforce the review’s warning about dependence on selected datasets.

The benchmark should also make its target explicit. Evaluating extraction from a disclosure differs from evaluating agreement with a commercial score. Transparent task definitions would help readers understand whether an improvement concerns text processing, provider imitation, or a separately justified assessment.

The second signal is a documented implementation of an integrated architecture under realistic reporting conditions. Such an evaluation would connect document preparation, concept mapping, modeling, and oversight. It would need to show how errors and uncertainty move between those components.

A study that isolates the effects of model upgrades and data preparation would be particularly informative. If stronger models overcome substantial reporting variation, that would qualify the review’s emphasis on infrastructure. If preparation and alignment dominate outcomes, that would strengthen its central interpretation.

The third signal is a public operational evaluation with independent or external expert involvement. Useful reporting would explain unresolved classifications, required corrections, and the relationship between outputs and source evidence. It would show the boundaries of operation alongside successful examples.

Evidence of dependable performance under those conditions would weaken concerns about limited deployment maturity. Repeated dependence on narrow data conditions would strengthen them. Either outcome would offer more information than a demonstration that provides polished answers without revealing how reviewers checked them.

For developers, these signals identify research opportunities rather than a requirement to wait for universal standardization. For buyers, they identify evidence that distinguishes a useful bounded application from a broader claim. For analysts, they preserve room for assistance while keeping substantive judgments open to inspection.

Read the Mathebula ESG AI review alongside the methods and evaluation sections of any system you are assessing. Compare the task actually tested with the decision your team expects to make. Ask whether the evidence covers unfamiliar reporting conditions and preserves disagreements between sources. Keep extraction accuracy separate from confidence in an overall sustainability judgment. A convincing result should make its supporting documents, definitions, and unresolved mappings available for inspection. That standard leaves room for useful progress without treating a fluent answer as a completed evaluation. When the next ESG AI performance claim arrives, examine what changed underneath the answer: the model, the data, the validation, or all three.

Get your work moving in remio

See what needs your attention and take the next step in Workspace.

remio currently supports Windows 10+ (x64) and Macs with Apple silicon.

Your AI Partner at Work
Get more done with remio

Plan. Create. Deliver.
All in one place.

bottom of page