Georeferencing complex relative locality descriptions with large language models

AI summary

Overview

Research area: Georeferencing (assigning geographic coordinates to text), natural language processing, large language models, and biodiversity informatics.

Technical level: Intermediate. The paper assumes some familiarity with concepts such as gazetteers, toponym resolution, prompt patterns, and parameter-efficient fine-tuning, but the core idea is explained in plain terms.

Scope: This paper investigates whether open-source large language models, fine-tuned with Quantized Low-Rank Adaptation (QLoRA), can automatically convert complex relative locality descriptions in biological specimen records into geographic coordinates.

What This Paper Is About

Many biological specimen records collected before GPS describe where a specimen was found in narrative language, using relative spatial phrases such as "10 km west of" a named place or "near" a creek, rather than giving coordinates. Existing georeferencing methods rely mostly on place names (gazetteer matching) or on models that associate words with regions, and they handle relative spatial relationships poorly, so the authors set out to test whether large language models can do the job. The goal is an automated georeferencing approach for the biodiversity collections domain that works across different regions and languages.

Key Contributions

1. The authors demonstrate the effectiveness of applying LLMs to georeferencing locality descriptions that contain relative spatial relationships, and establish a benchmark that they report significantly outperforms commonly used baselines.
2. They investigate which LLM prompting patterns work best for georeferencing and report that the Context Manager pattern is optimal for this task, yielding very good results.
3. They analyse the regional and linguistic generalizability of their LLM-based approach across datasets from the USA, New Zealand, Australia, and Mexico (the Mexico dataset being in Spanish).
4. They report that their model is more robust for non-English datasets compared to the baselines, and that for smaller regions such as New Zealand, a model fine-tuned with just 5,000 samples outperforms the baselines.

Main Findings

• Overall performance: Across datasets, the approach places an average of 65% of records within a 10 km radius, for a fixed amount of training data, outperforming existing baselines. The best results, reported for New York state, were 85% within 10 km and 67% within 1 km.
• New Zealand results: The fine-tuned LLM predicted 70.43% of localities within 10 km, 25.36% within 1 km, and 7.71% within 100 m, with a median Simple Accuracy Error (SAE) of 3.55 km and a mean SAE of 41.95 km. GEOLocate reached only 45.39% within 10 km and 10.45% within 1 km, with substantially higher error values, and the gazetteer method performed worst overall.
• Australia results: The fine-tuned LLM predicted 53.82% of records within 10 km and 9.8% within 1 km, with a median SAE of 8.52 km and a mean SAE of 51.28 km, again surpassing the baselines.
• USA results: GEOLocate outperformed the LLM on the USA dataset, predicting 26.82% of localities within a 1 km radius versus 7.40% for the LLM, with better median and mean SAE as well. The authors suggest this may relate to GEOLocate being intended for the USA, Canada, and Mexico, but note that their approach was superior on data for Mexico and for two states of the USA.
• Prompt pattern comparison: Testing Zero-shot, Zero-shot Chain of Thought, Chain of Thought, Context Manager (Context Control), and Persona patterns on the locality description "21 km west of Opotiki, south of Wainui Road" with ChatGPT-4 produced errors of 3.27 km (Zero-shot), 3.29 km (Chain of Thought), and 3.03 km (Context Manager); the Zero-shot Chain of Thought and Persona patterns generated no coordinates at all, returning step-by-step guidance instead.
• Model selection: Initial experiments compared Llama2-7B, Llama3-7B, Gemma-7B, and Mistral-7B base models, and identified Mistral-7B as the most suitable. Instruct versions of these models did not perform as well as the base models for this task.
• Where LLMs struggle: When the model lacks knowledge of the geographic coordinates of place names mentioned in a description, it did not produce coordinates and instead gave follow-up instructions.
• Long complex descriptions: The selected LLM is reported to perform well on lengthy, complex descriptions.

Methodology in Plain English

The authors pulled biological collection records from the Global Biodiversity Information Facility (GBIF) for four regions: the USA (New York Botanical Garden Herbarium), New Zealand (Manaaki Whenua – Landcare Research), Australia (Australasian Virtual Herbarium), and Mexico (National Biological Collections of the Institute of Biology, UNAM). The USA, New Zealand, and Australia sets are in English; the Mexico set is in Spanish. From each record they used the "locality" text as the input to georeference, and the original decimalLatitude and decimalLongitude as ground truth, with country and state or province as extra context. After removing duplicates and records without original coordinates, they randomly selected approximately 30,000 records per dataset for initial experiments, and each country dataset was split into 70% training, 15% validation, and 15% testing.

They first explored prompting by hand. Using ChatGPT-4 through the ChatGPT interface, they tried several prompt patterns and measured errors on sample localities. The Context Manager pattern, which supplies the country and state or province so the output is constrained to that region, was selected as the best fit, and the datasets were formatted in that style, with the original coordinates included in training and validation prompts as ground truth.

They then fine-tuned open-source models using QLoRA with 4-bit quantization, which let them train on a single NVIDIA A100 24 GB GPU. The key settings were a learning rate of 2e-4, batch size of 32, LoRA rank of 32, LoRA alpha of 64, and three training epochs. Fine-tuning was done separately for each region-specific dataset, producing a dedicated model per region.

For comparison, they used two baselines: GEOLocate (taking its highest-ranked coordinate pair) and a gazetteer-matching algorithm they implemented themselves. That baseline fine-tuned the spaCy model on 50 annotated locality descriptions for place name recognition, geocoded the names with GeoNames and Nominatim (specifying country and state or province for initial disambiguation), and used DBSCAN to find the densest cluster of candidate points, taking the mean latitude and mean longitude of that cluster. They also compared their model against GPT-5, GPT-4.1, and GPT-4o. Performance was measured with mean and median SAE (Simple Accuracy Error) and the percentage of predictions falling within 10 km and 1 km of the true location, using the Haversine formula for distance.

Why This Matters

Impact on research: Georeferenced specimen records are what allow researchers to map where species were found, monitor changes in geographic distribution over time, and study how environmental change affects biodiversity. Because most digitised records contain verbal locality descriptions rather than coordinates, and because manual georeferencing is slow (the paper cites 5 to 15 minutes per record for the point-radius method and 15 to 90 minutes for the shape method), automation at the scale of billions of records is a longstanding bottleneck. This work tests a new class of model for that task and reports the first use of LLMs to georeference text by predicting coordinates directly from locality descriptions, as far as the authors are aware.

Real-world applications:

• Populating coordinates for legacy museum and herbarium specimen records that predate GPS, enabling them to be mapped.
• Monitoring species distributions over time and the effects of environmental change on species.
• Regional biodiversity assessment and planning, including for smaller regions where only limited training data (such as 5,000 samples) is available.
• Extending georeferencing to non-English collections, demonstrated with a Spanish-language Mexican dataset.

Industry relevance: The approach uses open-source models and runs on a single 24 GB GPU, which the authors present as making the solution accessible without large-scale computing infrastructure. Commercial GPT models (GPT-5, GPT-4.1, GPT-4o) are included as comparison points, which matters for institutions deciding between paid APIs and self-hosted fine-tuned models. Existing tools such as GEOLocate, BioGeomancer (now non-functional) and GeoPick are discussed as the current landscape that this work competes with or complements; GeoPick is explicitly noted not to be an automatic georeferencing tool.

Future Directions

• Improving performance on USA-style data: Since GEOLocate beat the fine-tuned LLM on the USA dataset while the LLM was superior on Mexico and two USA states, understanding what drives those regional differences, and whether hybrid approaches could combine the strengths of both, is an open question.
• Handling unknown place names: The authors observed that when the model does not know the coordinates of place names in a description, it returns instructions instead of coordinates. Addressing this limitation is a natural next step.
• Extending regional and linguistic coverage: The study covers the USA, New Zealand, Australia, and Mexico; generalizability to other regions and languages remains to be tested at scale.
• Uncertainty reporting: The traditional point-radius method expresses an area of uncertainty around a coordinate, but the metrics reported here are error distances and percentages within radii; whether LLM-based georeferencing can output calibrated uncertainty alongside coordinates is not addressed in the content available.

Target Audience

This paper is most useful to researchers and practitioners in biodiversity informatics and natural history collections who need to georeference large volumes of legacy specimen records; to NLP and GIS researchers working on georeferencing, toponym resolution, and spatial language; and to developers evaluating whether fine-tuned open-source LLMs can replace or supplement gazetteer-based and rule-based geocoding tools such as GEOLocate. Readers with a background in machine learning or geospatial data will get the most out of the experimental detail, while the framing of the problem is accessible to collections managers and curators.

Authors’ abstract

Georeferencing text documents has typically relied on either gazetteer-based methods to assign geographic coordinates to place names, or on language modelling approaches that associate textual terms with geographic locations. However, many location descriptions specify positions relatively with spatial relationships, making geocoding based solely on place names or geo-indicative words inaccurate. This issue frequently arises in biological specimen collection records, where locations are often described through narratives rather than coordinates if they pre-date GPS. Accurate georeferencing is vital for biodiversity studies, yet the process remains labour-intensive, leading to a demand for automated georeferencing solutions. This paper explores the potential of Large Language Models (LLMs) to georeference complex locality descriptions automatically, focusing on the biodiversity collections domain. We first identified effective prompting patterns, then fine-tuned an LLM using Quantized Low-Rank Adaptation (QLoRA) on biodiversity datasets from multiple regions and languages. Our approach outperforms existing baselines with an average, across datasets, of 65% of records within a 10 km radius, for a fixed amount of training data. The best results (New York state) were 85% within 10km and 67% within 1km. The selected LLM performs well for lengthy, complex descriptions, highlighting its potential for georeferencing intricate locality descriptions.