EO-OS Object Detection#

This notebook demonstrates automated object detection from satellite imagery using the SDK with the EO-OS model to identify and locate objects at scale.

What This Does:#

The GeoAI SDK processes high-resolution aerial imagery through the EO-OS model to detect and geolocate 20+ object classes including:

  • ✈️ Airplanes and airports

  • πŸš— Vehicles (cars, trucks)

  • 🚒 Ships and harbors

  • 🏟️ Sports facilities (stadiums, courts, fields)

  • πŸ—οΈ Infrastructure (buildings, bridges, storage tanks, train stations)

Each detection includes precise coordinates, confidence scores, and class labels. Results are published to your GeoCatalog as searchable STAC items.

Workflow:#

  1. Define areas of interest (AOI) and time range

  2. SDK queries STAC catalogs for imagery

  3. Model processes imagery to detect objects

  4. Results published to GeoCatalog

Prerequisites:#

Before running this notebook:

  1. Deploy Model in Azure AI Foundry: Follow the Azure AI Foundry deployment guide to deploy an EOOS model and obtain your endpoint URL and API key

  2. Create Azure Storage Account: Set up a storage account with a blob container for storing results (e.g., sdk-results)

  3. Prepare GeoCatalog: Set up your GeoCatalog instance for publishing results to a collection

  4. Azure RBAC Roles: Ensure you have the required roles assigned:

    • Storage: Storage Blob Data Contributor + Storage Blob Delegator

    • Model endpoint: AzureML Data Scientist or Azure AI Developer

    • GeoCatalog: Write access to target collection

  5. Install SDK:

    # From wheel (download from GitHub Release)
    pip install "geoai_sdk-0.1.0-py3-none-any.whl[examples]"
    
    # Or from cloned repo
    pip install "tools/geoai-sdk[examples]"
    
    # Register Jupyter kernel
    python -m ipykernel install --user --name=geoai-sdk
    
  6. Azure Authentication: Configure Azure CLI

    az login
    
  7. Select Kernel: Choose β€œPython 3.9+ (geoai-sdk)” as the notebook kernel

  8. Configure Settings: Update cells with your model endpoint and storage details

Optional: Copy .env.example to .env (in the same directory as this notebook) to set environment variables

1. Setup and Imports#

The SDK automatically configures clean, production-ready logging (INFO level by default).

Logging Modes:

  • INFO (default): High-level progress only - clean logs for production

  • DEBUG: Detailed per-chip processing logs - useful for troubleshooting

To enable debug mode:

os.environ['GEOAI_LOG_LEVEL'] = 'DEBUG'  # Before importing geoai
import asyncio
import os
from pathlib import Path
from azure.identity import DefaultAzureCredential

# Load environment variables from .env file if it exists
try:
    from dotenv import load_dotenv
    env_path = Path(".env")
    if env_path.exists():
        load_dotenv(env_path)
        print("βœ… Loaded environment variables from .env")
except ImportError:
    pass  # python-dotenv not installed

import geoai

print("βœ… SDK loaded successfully!")
print(f"   Log level: {os.getenv('GEOAI_LOG_LEVEL', 'INFO')}")

2. Azure Authentication#

When you need Azure authentication:

  • πŸ“€ Publishing results to GeoCatalog (includes blob storage for assets)

  • πŸ”’ Using private GeoCatalog as input

# Initialize Azure credential
credential = DefaultAzureCredential()

# Optional: Load storage account key from environment
storage_account_key = os.getenv("STORAGE_ACCOUNT_KEY")

if storage_account_key:
    print("πŸ”‘ Using storage account key for SAS generation")
else:
    print("πŸ” Using Azure AD authentication (recommended)")
    print("   Ensure you have 'Storage Blob Delegator' role")

3. Discover Available Models (Optional)#

Check which models are available and what collections they support.

About Collections:

  • Planetary Computer: Pre-defined datasets like NAIP, Landsat, Sentinel-2

    • EOOS/MARS support: NAIP only (high-resolution 0.6m imagery)

  • Private GeoCatalog: Your own ingested high-resolution imagery

    • Use any collection name you want

# List all available models
print("πŸ“š Available Models:\n")
for model_info in geoai.models.list():
    print(f"  πŸ€– {model_info['model_name']} ({model_info['name'].upper()})")
    print(f"     Collections: {', '.join(model_info['supported_collections'])}")
    print(f"     Bands: {', '.join(model_info['required_bands'])}")
    print(f"     Resolution: {model_info['preferred_resolution_meters']}m")
    print()

# Get detailed information about EOOS
print("πŸ” EOOS Model Details:")
eoos_info = geoai.models.get("eoos")
print(f"   Supported collections: {eoos_info['data_requirements']['supported_collections']}")
print(f"   Required bands: {eoos_info['data_requirements']['required_bands']}")

4. Define Input Source#

Configure where to get satellite imagery.

Option 1: Public Planetary Computer (default - easiest!)

  • Just specify collection: naip

  • No authentication needed

  • NAIP: High-resolution aerial imagery (0.6m, USA)

Option 2: Private GeoCatalog (for your own data)

  • Ingest your high-resolution imagery as STAC items

  • Specify your GeoCatalog URI and collection name

  • Requires Azure authentication

# Option 1: Public Planetary Computer (default - simplified!)
input_source = geoai.Input(collection="naip")

# Option 2: Private GeoCatalog (uncomment and configure)
# input_source = geoai.Input(
#     collection="your-high-res-imagery",  # Your custom collection name
#     geocatalog_uri="https://your-company.geocatalog.com/stac",
#     credential=credential  # Azure credential required
# )

print(f"πŸ“₯ Input configured: {input_source.collection}")
print(f"   Source: {input_source.geocatalog_uri}")

5. Define Areas of Interest (Multi-AOI)#

This example processes multiple locations simultaneously using a GeoParquet file.

Example File: test_airports_eoos.geoparquet

  • Contains 3 airport AOIs for demonstration:

    • LAX - Los Angeles International Airport

    • SFO - San Francisco International Airport

    • SEA - Seattle-Tacoma International Airport

  • Each AOI covers the airport area with runways and terminals

  • Shows how to process multiple locations in a single run

Multi-AOI Benefits:

  • 🌍 Process many locations in one run

  • πŸ“Š Efficient STAC search (SDK handles optimization)

  • πŸ“€ All results published to single collection

  • ⚑ Concurrent chip processing across all AOIs

NAIP Update Cycle:

  • NAIP imagery is collected approximately every 3 years

  • For best data availability, use datetime ranges 3+ years apart (e.g., β€œ2017-01-01/2020-12-31” or β€œ2020-01-01/2023-12-31”)

  • Narrow date ranges may result in no imagery coverage for some AOIs

Tips:

  • Use https://geojson.io to draw custom AOIs

  • Save as GeoJSON or GeoParquet

  • Each AOI processed independently (failures don’t affect others)

# Create multi-AOI constraint from GeoParquet file
# SDK accepts file path directly - no need to load with geopandas first!
constraint = geoai.Constraint(
    aois="test_airports_eoos.geoparquet",  # Pass file path directly
    datetime="2020-01-01/2023-12-31"  # NAIP updates ~every 3 years - use 3+ year range for best coverage
)

print(f"βš™οΈ  Constraint configured:")
print(f"   Mode: {constraint.mode}")  # Should show "multi"
print(f"   AOIs: {len(list(constraint.iter_aois()))}")
print(f"   Datetime: {constraint.datetime}")

# Optional: Preview what's in the file (for inspection only)
import geopandas as gpd
aois_preview = gpd.read_parquet("test_airports_eoos.geoparquet")
print(f"\nπŸ“ Processing {len(aois_preview)} AOIs:")
for idx, row in aois_preview.iterrows():
    area_km2 = row.geometry.area * 111 * 111  # Rough conversion
    print(f"   {row['id'].upper()}: {row['name']} (~{area_km2:.1f} kmΒ²)")

# Alternative: Single AOI from bbox (uncomment to use instead)
# from shapely.geometry import box
# constraint = geoai.Constraint(
#     bbox=[-118.42270374298097, 33.93845399220425, -118.40141773223878, 33.94649511083257],
#     datetime="2020-01-01/2023-12-31"  # NAIP: Use 3+ year range
# )

# Optional: Filter by imagery resolution (GSD - Ground Sample Distance)
# NAIP example - find high-resolution imagery only:
# constraint = geoai.Constraint(
#     bbox=[-118.42, 33.938, -118.401, 33.946],
#     datetime="2020-01-01/2023-12-31",
#     filter={"gsd": {"lte": 0.5}}  # Only imagery 0.5m/pixel or better
# )

# Note: Cloud cover filters only work with satellite imagery (Sentinel, Landsat)
# NAIP is aerial imagery without cloud metadata, so cloud filters are not applicable
# For satellite imagery, combine GSD and cloud cover:
# filter={"gsd": {"lte": 10}, "eo:cloud_cover": {"lt": 5}}  # Sentinel/Landsat only

5a. Advanced: Custom STAC Search (Optional)#

Skip this section if you’re using the simple constraint above.

For power users who need full control over imagery selection (sorting, limiting, complex filters), you can build a custom STAC search and pass it directly to the SDK.

When to use:

  • 🎯 Sort by date (latest first) or cloud cover (lowest first)

  • πŸ“Š Limit to N best items (testing, cost control)

  • πŸ” Multi-AOI efficiency (one search for many locations)

  • βš™οΈ Complex CQL2 filter expressions

How it works:

  • You build the search with pystac_client

  • SDK uses your search results for all AOIs

  • AOIs without imagery in search results are skipped with warning

# OPTIONAL: Build custom STAC search for advanced control
# Uncomment to use this instead of the simple constraint above

# import pystac_client
# 
# # Build custom search with advanced options
# client = pystac_client.Client.open("https://planetarycomputer.microsoft.com/api/stac/v1")
# 
# custom_search = client.search(
#     collections=["naip"],
#     bbox=[-118.42270374298097, 33.93845399220425, -118.40141773223878, 33.94649511083257],
#     datetime="2020-01-01/2023-12-31",
#     
#     # Advanced options:
#     query={"eo:cloud_cover": {"lt": 5}},  # Very low cloud cover
#     sortby=[{"field": "properties.datetime", "direction": "desc"}],  # Latest first
#     limit=10  # Only top 10 items
# )
# 
# # Replace simple constraint with advanced version
# constraint = geoai.Constraint(
#     bbox=[-118.42270374298097, 33.93845399220425, -118.40141773223878, 33.94649511083257],
#     stac_search=custom_search  # SDK will use your custom search
# )
# 
# print(f"πŸ“ Advanced constraint configured")
# print(f"   Using custom STAC search with sorting/limiting")

# For multi-AOI: Search large area once, process many AOIs efficiently!
# search = client.search(bbox=[-118.5, 33.9, -118.3, 34.1], limit=50)  # All of LA
# constraint = geoai.Constraint(aois="buildings.geoparquet", stac_search=search)

6. Configure Output Destination#

All results are published to GeoCatalog with assets stored in Azure Blob Storage.

Required Setup:

  1. GeoCatalog: Your GeoCatalog endpoint URL and collection name

  2. Azure Blob Storage: Storage account and container for chip imagery and STAC item assets

    • Create a storage account

    • Create a container (e.g., β€œsdk-results”)

  3. Authentication (choose one):

    • Recommended: Azure RBAC roles (no key needed)

      • Storage Blob Data Contributor

      • Storage Blob Delegator

    • Alternative: Set STORAGE_ACCOUNT_KEY environment variable

      export STORAGE_ACCOUNT_KEY="your_key_here"
      

Optional: Enable save_local=True to save GeoJSON files locally for debugging.

Update the values below with your settings:

# Configure output destination (GeoCatalog + Blob Storage)
output = geoai.Output(
    # GeoCatalog endpoint (required)
    geocatalog_uri="https://your-geocatalog.example.com/",
    collection_name="my-eoos-detections",
    credential=credential,
    
    # Blob storage for chip imagery and STAC assets (required)
    storage_url="https://yourstorageaccount.blob.core.windows.net",
    blob_container="sdk-results",
    storage_account_key=os.getenv("STORAGE_ACCOUNT_KEY"),  # Optional: use Azure AD if not provided
    
    # Run identifier (optional - auto-generates UUID if not provided)
    run_id=None,  # Auto-generate unique ID
    
    # Local file saving (optional - for debugging only)
    save_local=True,  # Set to True to save GeoJSON files locally
    output_dir="./output"  # Only used if save_local=True
)

print(f"πŸ“€ Output configured:")
print(f"   GeoCatalog: {output.geocatalog_uri}")
print(f"   Collection: {output.collection_name}")
print(f"   Blob container: {output.blob_container}")
print(f"   Run ID: {output.run_id or 'auto-generate'}")
if output.save_local:
    print(f"   Local output: {output.output_dir}/")

7. Initialize EO-OS Model#

Configure the model endpoint. Supports both Azure ML and Azure AI Foundry deployments.

Authentication Options:

Method

Use Case

Setup

API Key/Token

Simple, works everywhere

Get from Azure ML Studio or AI Foundry portal

Azure AD ⭐

Production, no secrets in code

Requires role assignment (see below)

Required Azure Roles for Azure AD:

  • Azure ML: AzureML Data Scientist role on workspace

  • AI Foundry: Azure AI Developer or Cognitive Services User role on project

Assign roles:

# Azure ML
az role assignment create --assignee user@contoso.com --role "AzureML Data Scientist" \
  --scope /subscriptions/{sub-id}/resourceGroups/{rg}/providers/Microsoft.MachineLearningServices/workspaces/{workspace}

# AI Foundry
az role assignment create --assignee user@contoso.com --role "Azure AI Developer" \
  --scope /subscriptions/{sub-id}/resourceGroups/{rg}/providers/Microsoft.CognitiveServices/accounts/{ai-service}
# Initialize EOOS model with your endpoint
# Authentication: Uses EOOS_API_KEY from .env if set, otherwise uses Azure AD (credential)

model = geoai.models.EOOS(
    endpoint="https://your-eoos-endpoint.eastus2.inference.ml.azure.com/score",
    credential=os.getenv("EOOS_API_KEY") or credential,  # API key from .env or Azure AD
    # num_instances=1,           # Configure based on your endpoint capacity
    # concurrent_per_instance=1  # Configure based on your endpoint capacity
)

print("πŸ€– EO-OS model initialized")
print(f"   Endpoint: {model.endpoint}")
if isinstance(model.credential, str):
    print(f"   Auth: API Key (from EOOS_API_KEY env var)")
else:
    print(f"   Auth: Azure AD (DefaultAzureCredential)")

8. Configure Detection Parameters#

Set model-specific parameters for EO-OS object detection.

Parameters:

  • chip_size: Size of image chips in pixels (default: 1024, max: 2048) - SDK-internal parameter

  • stride: Overlap between chips in pixels (default: 800) - SDK-internal parameter

  • threshold: Confidence threshold (0-1, higher = fewer but more confident detections) - sent to model endpoint

πŸ’‘ Recommended: Default parameters are pre-optimized for accuracy and speed.

# Detection parameters
params = {
    "chip_size": 1024,    # pixels (common: 512, 1024, 2048)
    "stride": 800,       # pixels (no overlap when stride = chip_size)
    "threshold": 0.3      # confidence threshold (0.0 - 1.0)
}

print(f"βš™οΈ  Detection parameters:")
print(f"   Chip size: {params['chip_size']}px")
print(f"   Stride: {params['stride']}px")
print(f"   Threshold: {params.get('threshold', 'not set')}")

10. Run Object Detection#

Execute the detection workflow. This may take several minutes depending on AOI size.

Workflow Steps:

  1. πŸ”’ Validation

  2. πŸ”² Chip creation

  3. πŸ” STAC search

  4. πŸ“₯ Image fetching

  5. πŸ€– Model inference

  6. πŸ”„ Result merging

  7. πŸ“€ GeoCatalog publishing

# Run object detection with publishing
# SDK handles all logging internally
result = await model.run(
    input=input_source,
    constraint=constraint,
    params=params,
    output=output
)

11. View Results#

Review detection statistics and output locations.

Results include:

  • Detection count and chip statistics

  • Local GeoJSON file with all detections

  • GeoCatalog publication link (if enabled)

  • Blob storage location for chip imagery

# Print results summary
print("\n" + "="*70)
print("πŸ“Š EO-OS Detection Results")
print("="*70)

# GeoCatalog Published section first
if result.published:
    print(f"\n☁️  GeoCatalog Published:")
    if result.total_aois:
        # Multi-AOI: show collection-level info
        collection_url = f"{output.geocatalog_uri.rstrip('/')}/collections/{output.collection_name}"
        print(f"   Collection: {collection_url}")
        print(f"   Items published: {result.successful_aois} AOIs")
        
        # Extract run_id from first AOI result (all AOIs share same run_id)
        workflow_run_id = output.run_id  # User-provided run_id (if any)
        if not workflow_run_id and hasattr(result, '_raw_result') and 'results' in result._raw_result:
            # SDK stores run_id in each AOI result
            for aoi_result in result._raw_result['results']:
                if 'result' in aoi_result:
                    workflow_run_id = aoi_result['result'].get('run_id')
                    if workflow_run_id:
                        break
        
        if workflow_run_id:
            print(f"   Run ID: {workflow_run_id}")
            print(f"   Blob storage: {output.storage_url}/{output.blob_container}/eoos/{workflow_run_id}/")
            print(f"      └── Results in: eoos/{workflow_run_id}/results/<aoi_id>/")
        else:
            print(f"   Run ID: (not available)")
            print(f"   Blob storage: {output.storage_url}/{output.blob_container}/")
        
        print(f"\n   πŸ’‘ View individual AOI results in the GeoCatalog collection")
    else:
        # Single AOI results
        collection_url = f"{output.geocatalog_uri.rstrip('/')}/collections/{output.collection_name}"
        print(f"   Collection: {collection_url}")
        
        # Get run_id from output (user-provided) or from result
        workflow_run_id = output.run_id
        if not workflow_run_id and hasattr(result, '_raw_result'):
            workflow_run_id = result._raw_result.get('run_id')
        
        if workflow_run_id:
            print(f"   Run ID: {workflow_run_id}")
            print(f"   Blob storage: {output.storage_url}/{output.blob_container}/eoos/{workflow_run_id}/")
        else:
            print(f"   Run ID: (not available)")
            print(f"   Blob storage: {output.storage_url}/{output.blob_container}/")
else:
    print(f"\n⚠️  GeoCatalog publishing: Not enabled or failed")

# Local output section (only shown if save_local=True)
if output.save_local:
    print(f"\nπŸ’Ύ Local Output:")
    if result.total_aois:
        # Multi-AOI: each AOI has its own subdirectory
        print(f"   Directory: {output.output_dir}/")
        print(f"   Structure:")
        print(f"      {output.output_dir}/")
        print(f"      β”œβ”€β”€ <aoi_id>/")
        print(f"      β”‚   β”œβ”€β”€ detections.geojson  (filtered to AOI bounds)")
        print(f"      β”‚   └── final_overlay.jpg   (visual overlay)")
    else:
        # Single AOI: files in output_dir
        print(f"   Directory: {output.output_dir}/")
        print(f"   Files: detections.geojson, final_overlay.jpg")

# Check if multi-AOI or single AOI
if result.total_aois:
    # Multi-AOI results
    print(f"\n🌍 Multi-AOI Processing:")
    print(f"   Total AOIs: {result.total_aois}")
    print(f"   Successful: {result.successful_aois}")
    print(f"   Failed: {result.total_aois - result.successful_aois}")
    
    print(f"\nπŸ“ˆ Aggregate Statistics:")
    print(f"   Total chips: {result.total_chips}")
    print(f"   Successful: {result.successful_chips}")
    print(f"   Total detections: {result.detection_count}")
    
    # Show detection categories if available (for multi-class models)
    print(f"\nπŸ“Š Detections by Category:")
    aoi_results = result._raw_result.get('results', [])
    if aoi_results:
        category_totals = {}
        for aoi_result in aoi_results:
            aoi_data = aoi_result.get('result', {})
            if 'detection_counts' in aoi_data:
                for category, count in aoi_data['detection_counts'].items():
                    category_totals[category] = category_totals.get(category, 0) + count
        
        if category_totals:
            for category, count in category_totals.items():
                print(f"   β€’ {category}: {count}")
        else:
            print(f"   Total: {result.detection_count}")
    else:
        print(f"   Total: {result.detection_count}")
    
    print(f"\nπŸ“ Per-AOI Breakdown:")
    aoi_results = result._raw_result.get('results', [])
    if aoi_results:
        for aoi_result in aoi_results:
            aoi_id = aoi_result.get('aoi_id', 'unknown')
            aoi_data = aoi_result.get('result', {})
            print(f"   {aoi_id.upper()}:")
            print(f"      Chips: {aoi_data.get('total_chips', 0)}")
            print(f"      Detections: {aoi_data.get('detection_count', 0)}")
else:
    # Single AOI results
    print(f"\nπŸ“ˆ Processing Summary:")
    print(f"   Total chips: {result.total_chips}")
    print(f"   Successful: {result.successful_chips}")
    print(f"   Failed: {result.total_chips - result.successful_chips}")
    
    print(f"\n🎯 Detections:")
    print(f"   Objects detected: {result.detection_count}")
    print(f"   Avg per chip: {result.detection_count / result.successful_chips:.1f}" if result.successful_chips > 0 else "   Avg per chip: 0")
    
    # Show detection categories if available (for multi-class models)
    aoi_data = result._raw_result.get('results', [{}])[0].get('result', {}) if hasattr(result, '_raw_result') else {}
    if 'detection_counts' in aoi_data and len(aoi_data['detection_counts']) > 1:
        print(f"\n   By Category:")
        for category, count in aoi_data['detection_counts'].items():
            print(f"      β€’ {category}: {count}")

print("\n" + "="*70)

12. Next Steps#

View Results:

  • Open GeoCatalog URL above to visualize detections

  • Check ./output/ directory for local files and visualizations

  • Load GeoJSON in QGIS, ArcGIS, or other GIS tools

Process More Locations:

  • Add more AOIs to test_aois.geoparquet file

  • Create your own GeoParquet with custom locations

  • Use geopandas to filter/subset AOIs for processing

Adjust Parameters:

  • Threshold (Cell 8): Increase to reduce false positives, decrease for more detections

  • Chip size (Cell 8): Larger chips (2048px) for better context, smaller (512px) for detail

  • AOIs (Cell 5): Add/remove locations, change datetime range

Before Large Jobs:

  • Always run estimation (Cell 9) to verify data availability

  • Check chip count and adjust stride if needed

  • Test on small AOI subset first to validate parameters

Multi-AOI Tips:

  • SDK processes each AOI independently

  • Failed AOIs don’t stop the entire job

  • All results published to single GeoCatalog collection

  • Check per-AOI breakdown in results

Troubleshooting:

  • No detections: Lower threshold, check AOI has visible objects

  • Too many false positives: Increase threshold (try 0.3-0.5)

  • Authentication errors: Run az login and verify RBAC roles

  • GeoCatalog errors: Check collection exists and credentials have write access

  • Some AOIs skipped: Check STAC item coverage, expand datetime range


πŸ“š Additional Resources#

Documentation:

  • SDK Documentation - Complete SDK reference

  • MARS Notebook - Building/road detection

External Resources:

SDK Features:

  • πŸ” Discovery: models.list(), models.get(), Model.get_supported_collections()

  • πŸ”’ Validation: Automatic input/collection validation before processing

  • πŸ“Š Estimation: model.estimate() - Preview chip count and verify data

  • πŸš€ Execution: model.run() - Process with automatic validation