Risks and Realities of Explainability in AI & Geospatial
Large language models (LLMs) and vision language models (VLMs) in geospatial and remote sensing workflows has both evolved and continues to move at pace. Advances in AI workflows, alongside expanding satellite data archives, and cloud-based processing are enabling Earth observation platforms to power wider business information. These platforms now allow organisations to derive insight from increasingly complex geospatial datasets with limited baseline knowledge.
Earth observation intermediaries and analytics providers can now provide layered infrastructures on technologies like Google Earth Engine. They enable enterprise end-users to work with space-based datasets at scale. The end-user no longer requires deep geospatial or remote sensing expertise. The new, lower technical threshold allows these enterprise users to engage with vast Earth observation archives through intuitive interfaces and modular workflows. And users implicitly trust these components, why wouldn’t they? The result resembles business to business software services, like the roles played by Xero in accounting or Figma in design. This shift has opened new opportunities for building, interpreting, and deploying geospatial intelligence. It is particularly relevant to climate risk, land use monitoring, and environmental compliance.
But this new accessibility brings with it an overlooked challenge: Explainability.
Specificity to Generalised Models
The use of AI in spatial analytics is not new. As early as 2015, machine learning (ML) classifiers helped to computationally exploit LiDAR data from rail networks. These models could detect and map signal boxes, trackside equipment, electrical systems, and catenary infrastructure acquired using UAV platforms and train-mounted sensors. These early applications were accurate and consistent, operating in well-defined domains where training data matched deployment conditions. At the same time, these same ML classifiers applied to tram systems in urban areas, required only small and few developmental changes. These subtle changes were necessary to account for city trees and other variables. Importantly, these classifiers were designed to only detect and represent a fraction of the total landscape.
Grounded in narrow, controlled environments a decade ago, Earth observation platforms now operate in a vastly different context. They span variable landscapes and infrastructure types, use different sensors, and operate at varying resolutions and timescales. Today’s AI analytics companies present outputs with a strong apparent visual appeal and confidence. The risk is that these digital outputs become mistaken by users as direct explanations of what is happening in the real world, when in fact they may only be partially valid or accurate in select cases.
Explainability
In geospatial AI, explainability (to be explanatory) means more than just making a model transparent or repeatable. It means being able to interpret outputs in terms of physical processes and real-world contexts e.g. is your digital twin an accurate reflection of reality. This includes understanding the assumptions, missing inputs, and environmental drivers behind a model's output. Sometimes these drivers are known and clearly described. Other times, analytics providers weakly approximate assumptions or leave key inputs unknown, yet the output remains framed as if it were definitive and comprehensive.
Take a system that calculates NDVI or Leaf Area Index over farmland. The case study data might be cloud free, captured at the right times relative to growing seasons, and processed correctly. The output may show patterns in plant health or field variation. But if that system aims to infer yield, then those missing variables require integration too.
These parameters include crop type, soil conditions, slope, local weather, sunlight, disease, and key land management interventions, either directly measured or inferred.
Without those inputs, output maps may look convincing, but they cannot fully explain what is happening. The analysis becomes suggestive and is neither definitive nor comprehensive. This gap is the core issue of explainability.
Lowering Barriers & Enhancing Responsibility
The same tools that have helped reduce the technical skill required to work with satellite data now require additional supporting information and rigour. However, end-user consumption of building blocks and transformers, curated datasets, and traceable data sources allows users to access insights that were once available only to specialists.
This is a major development. It allows a variety of organisations to bring satellite data into decisions far more easily. These tools now function similarly to other enterprise services. Satellite data has become a commodity input, akin to financial or compliance data in other domains.
But the ease of use creates this new risk. Outputs that appear complete may not fit their intended use. They may miss key assumptions, oversimplify complex systems, or exist as evidence without contemplation of the embedded risk and caveats. This is why expert input still matters. You might use Xero every day, but you would still ask an accountant for help with your tax planning. The same logic applies here.
These tools look elegant and are easy to use, but they may hide the uncertainty behind the results. That complexity still exists, even if it is not visible.
Complexity & Failure Modes
The combined use of AI, environmental science, and physics brings power, but it also carries risk. Each of these areas introduces its own failure points.
Sensor and Data Fragility: Satellite analysis depends on the type of sensor, its orbit, the continuity of data, and the conditions on the ground. An AI methodology that works for one satellite may not work for another.
Model Generalisation Limits: AI models trained in one place may not work well elsewhere. Vision language models can struggle to adapt unless retrained. Claims that a model works globally probably ought to be carefully considered. Training one region takes hundreds of examples. Training the world could take millions.
Environmental Complexity: Real world phenomena may involve undisclosed parameters, some of which cannot be easily observed from space. They instead require models and assumptions that, even when well-built, carry residual uncertainty.
These problems become worse when software providers overstate what their tools can do. There is a risk that outputs available to unsuspecting end-users are described as comprehensive explanations, when they are only, at-best, indicators. Even if a map looks detailed and matches expectations, it may still miss critical variables.
Caution and Rigour
As large language models and vision models become mainstay to Earth observation systems, and as satellite data becomes more like other enterprise software services, users must treat these risks like any other business partnership or procurement risk.
Explainability is more than a technical feature. It is part of what makes an output trustworthy. It is what separates insight from illusion. There is danger in assuming that a clear, detailed AI-generated “scientific” output is automatically correct.
It is tempting to see high resolution as high accuracy and treat AI on scientific data as “cleverer than me.” That leap as a default step is prone to risk, so guard-rails are necessary.
With a cautious set of steps and protections, geospatial AI can deliver genuine insight, not false confidence.
Callala is collaborating with Airside Labs on the development of AI Benchmarks as guardrails to support safer use of LLMs and VLMs - https://airsidelabs.com/ai-aviation-eval-working-group/
*Graphic Design: Victoria Beall
