Towards Human-AI Accessibility Mapping in India: VLM-Guided Annotations and POI-Centric Analysis in Chandigarh

AI summary

Overview

Research area: Human-Computer Interaction, specifically crowdsourced urban accessibility mapping and human-AI annotation support.

Technical level: Intermediate — the paper assumes familiarity with crowdsourcing platforms and vision-language models, but explains its custom accessibility metric and adaptation process in accessible terms.

Scope: The paper describes adapting the Project Sidewalk sidewalk-auditing platform for deployment in Chandigarh, India, adding VLM-generated annotation guidance evaluated by three annotators, and using the adapted tool to run a Points-of-Interest (POI)-centric accessibility analysis across three sectors.

What This Paper Is About

Project Sidewalk lets people audit sidewalk accessibility by virtually walking through Google Street View imagery and labeling barriers such as missing curb ramps, uneven surfaces, and obstacles. It has been deployed in cities across the US, Mexico, Chile, and Europe, but its labels, examples, and underlying assumptions do not match Indian streetscapes, where pedestrians often walk on shared carriageways, informal shoulders, or discontinuous footpaths. This paper adapts the platform for India — changing labels and example imagery and adding AI-generated, per-segment mission guidance using a visual-language model — and then applies the adapted tool to analyze accessibility around roughly 230 POIs in three Chandigarh sectors with different land uses.

Key Contributions

1. Localized Project Sidewalk for India. The authors modified the interface for Chandigarh: the US "Curb Ramp" label was replaced with a broader "Curb Style" category, other labels had tags added or removed (for example, India-specific Obstacle in Path tags such as parked cars, carts, drainage, and electric boxes; removed tags such as fire hydrants and mailboxes), and all example images in hover tooltips were replaced with images from Chandigarh streets.

2. VLM-assisted mission guidance. A gemini-2.5-flash model generates short, context-aware annotation instructions from the OpenStreetMap road type of a segment plus its first and last Google Street View panoramas. Guidance triggers when a mission begins, when the annotator moves to a new street segment, and when the annotator uses the "Jump" option. The system does not create labels; it orients annotators toward relevant label categories.

3. Evaluation of AI guidance with three annotators. Fifty street segments were rated by three annotators on relevance, accuracy, and usefulness using a 5-point Likert scale, with reported means, Spearman correlations, and quadratic-weighted Cohen's kappa.

4. POI-centric accessibility analysis of three Chandigarh sectors. Using the adapted tool, the authors audited about 40 km of sidewalks around approximately 230 POIs in Sector 45 (residential), Sector 34 (commercial), and Sector 12 (institutional), and reported segment-level, POI-level, and POI-across-sector accessibility scores.

Main Findings

• VLM guidance scored highest on relevance. Across 150 ratings (50 segments x 3 annotators), Relevance had a mean of 4.97 (SD 0.26, min 2, max 5), Accuracy a mean of 4.40 (SD 0.71, min 2, max 5), and Usefulness a mean of 4.61 (SD 0.70, min 1, max 5). The abstract and discussion report an average utility score of 4.66.

• Annotator agreement varied by criterion. For Relevance, R1–R3 had perfect agreement (Spearman ρ = 1.000, p < 10⁻⁶, weighted κ = 1.00), while pairs involving R2 were non-computable due to constant ratings (κ = 0.00). For Accuracy, R2–R3 reached moderate agreement (κ = 0.49, ρ = 0.444, p = 0.0012), R1–R3 was fair (κ = 0.378, ρ = 0.384, p = 0.0059), and R1–R2 was weak (κ = -0.113, ρ = -0.209, p = 0.146). For Usefulness, R2–R3 showed substantial agreement (κ = 0.665, ρ = 0.445, p = 0.0012), while R1's agreement with others was lower.

• 1,644 of 2,913 locations flagged for improvement. Across 40 km of roads in three sectors and around 230 POIs, the authors identified 1,644 locations where infrastructure improvements could enhance accessibility.

• A long negative tail in segment scores. Segment accessibility scores showed many segments with substantial problems; scores were clipped at the 95th percentile for negative outliers, standardized, and passed through a sigmoid, where 0 is least accessible and 1 is most accessible.

• Sector 45 (residential) led on several POI categories. It showed the highest scores for residential, religious, social, and commercial POIs, with other categories needing attention.

• Sector 12 (institutional) was strongest for healthcare. Accessibility was best around healthcare in Sector 12, but other categories — especially utilities — were weak and need work.

• Sector 34 (commercial) showed moderate access for commercial POIs. Improvement was needed in other categories.

• Education and transport scored lowest across sectors. The authors state these categories should be prioritized. Commercial areas are described as having the best overall access, while education and public service lag.

• Functional accessibility is uneven. Healthcare access in the health-institution sector has received attention, but other facilities such as transit stops and food joints are not as accessible in the region.

• Platform scale as reported. The abstract states Project Sidewalk has been used in 40 cities worldwide; the introduction states it has grown to 44 cities across 10 countries, with over 10k users contributing 1.4 million labels assessing 26k km of city streets.

Methodology in Plain English

Selecting sectors. The team used Chandigarh's Master Plan land-use zoning plus ward-wise population data mapped to sector level (assuming uniform population within each ward). Sector 45 had the highest residential population. For the institutional category they compared Sector 12 (hospital area), Sector 1 (court complex), and Sector 14 (university campus), choosing Sector 12 because the PGIMER institution attracts large resident and traveling populations; that sector spans approximately 277 acres (1.12 sq km). For commercial areas they used Foursquare Places data across Sectors 26, 34, and 43, counting 162, 171, and 164 commercial POIs respectively, and selected Sector 34 as having the most.

Finding POIs and walkable paths. Each sector was divided into small spatial segments with one point sampled from each. The Google Places API collected all POIs within 400 m of each sampled point, initially yielding 23,136 POI entries; after removing duplicates by latitude and longitude, 10,128 unique POIs remained. From these, 98 distinct POI types were identified and grouped into 10 categories using OSM-style tags. Walking paths were traced from each POI up to 1 km in all directions with a depth-first search over a road graph built with OSMnx. Google Street View coverage was checked by dividing each 1 km buffer into small square cells (about 60–80 m), querying the nearest available GSV location at each cell centroid, and merging valid cells into a coverage area. Only path segments where at least 75% of the geometry overlapped with Street View coverage were kept.

Adapting the interface. Because "Curb Ramp" did not capture the range of Indian curb conditions — from formal concrete ramps to improvised slopes, stepped transitions, or partial modifications — it was renamed "Curb Style." Other labels kept their names but gained or lost tags to match Indian streetscapes, and all tooltip images were swapped for Chandigarh examples. The low/medium/high severity structure was preserved.

Adding AI guidance. The Gemini gemini-2.5-flash model receives the OSM road type (for example, residential, secondary, tertiary) and the segment's first and last Street View panoramas, along with a structured prompt asking for practical, India-aware annotation advice. The rationale is that residential roads often lack dedicated sidewalks while arterial roads more often have raised walking space or marked crossings, so the guidance adapts accordingly. Output appears in a popup and a persistent status panel above the minimap.

Scoring accessibility. Each labeled segment becomes a feature vector, with each label weighted between 0.2 and 1.0 converted from severity ratings 1–3 (0.2 for severity 1, 0.6 for severity 2, 1.0 for severity 3). Positive features such as curb ramps and marked crossings increase the score; negative ones such as surface problems reduce it. The segment score is a sigmoid of the dot product of the weight and feature vectors. POI-level scores are length-weighted averages of segment scores for all segments within a 1 km radius in the same sector. Across-sector scores are weighted averages of POI-level scores, weighted by the number of POIs of each type.

A note on labeling. "No sidewalk" was marked only when the sidewalk at that specific panorama was disrupted, broken, or missing — a local label, not a statement that the whole path lacks a sidewalk.

Why This Matters

Impact on research. This is described as the first deployment of Project Sidewalk in India. It extends prior geolocated accessibility mapping work — largely focused on US and European contexts — to a setting where pedestrian infrastructure takes fundamentally different forms, and it tests a specific human-AI interaction pattern: using a VLM to orient human annotators rather than to generate labels directly. The authors position this alongside prior LLM annotation-guidance work, noting that measuring label agreement directly is harder here because annotators must parse 3D street view space, so they use Likert-scale utility ratings instead.

Real-world applications:

• Municipal prioritization. The POI-centric framing targets places rather than all infrastructure, giving city bodies a first level of prioritization for where accessibility investment yields the most benefit.
• User-facing trip planning. Flagging accessible areas helps potential pedestrians understand where accessible routes exist.
• Infrastructure remediation targeting. The 1,644 flagged locations across 40 km provide a focused set of accessibility gaps for local action.
• Cross-sector comparison. Comparing residential, commercial, and institutional sectors highlights which land uses and facility types are lagging.

Industry relevance. The work is relevant to civic technology platforms, mapping and geospatial data providers, urban planning consultancies, and accessibility compliance efforts under India's RPwD Act 2016 and MoHUA Harmonised Guidelines for Barrier-Free Built Environment. The paper cites the scale of the problem: a Supreme Court Committee audit of Delhi's 1,400 km of PWD roads found 84% of footpaths failed to meet IRC standards and only 25% were usable, while Bengaluru walkability data from 2023 showed pedestrians made up 32% of all road deaths.

Future Directions

• Improving guidance precision. Accuracy and usefulness ratings showed more variation than relevance, so the authors state the guidance can be made more precise — particularly where a few messages were less precise or helpful.
• Extending beyond Chandigarh. The authors frame this as "the start of a journey for accessibility mapping in multiple Indian cities," implying replication across other Indian urban contexts with different infrastructure.
• Addressing lagging POI categories. Education and transport scored lowest across sectors, and in the institutional sector everyday destinations such as transit stops and food outlets were less accessible than healthcare — an open question is how to bring those up.
• Resolving annotator disagreement. R1 varied more than R2 and R3 on accuracy and usefulness, raising questions about how consistently different annotators interpret guidance and severity in ambiguous streetscapes.

Target Audience

This paper benefits HCI and accessibility researchers working on crowdsourced data collection and human-AI collaboration; urban planners and municipal accessibility practitioners in India and other low- and middle-income contexts; geospatial and civic technology developers adapting mapping tools across regions; and researchers interested in how vision-language models can support — rather than replace — human annotators on spatially complex labeling tasks.

Authors’ abstract

Project Sidewalk is a web-based platform that enables crowdsourcing accessibility of sidewalks at city-scale by virtually walking through city streets using Google Street View. The tool has been used in 40 cities across the world, including the US, Mexico, Chile, and Europe. In this paper, we describe adaptation efforts to enable deployment in Chandigarh, India, including modifying annotation types, provided examples, and integrating VLM-based mission guidance, which adapts instructions based on a street scene and metadata analysis. Our evaluation with 3 annotators indicates the utility of AI-mission guidance with an average score of 4.66. Using this adapted Project Sidewalk tool, we conduct a Points of Interest (POI)-centric accessibility analysis for three sectors in Chandigarh with very different land uses, residential, commercial and institutional covering about 40 km of sidewalks. Across 40 km of roads audited in three sectors and around 230 POIs, we identified 1,644 of 2,913 locations where infrastructure improvements could enhance accessibility.