geo-clip
This is an official PyTorch implementation of our NeurIPS 2023 paper "GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-
What is geo-clip?
This is an official PyTorch implementation of our NeurIPS 2023 paper "GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization"
Overview of geo-clip
What it is
GeoCLIP is an official PyTorch implementation of a NeurIPS 2023 paper titled “GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization.” It is a method for worldwide image geo-localization that aligns images with geographical locations using a CLIP-inspired contrastive learning approach.
Key features
- Uses a location encoder that models the Earth as a continuous function, learning CLIP-aligned features suitable for geo-localization.
- Trained contrastively on the MP-16 dataset, which contains 4.7 million images from across the globe.
- Achieves state-of-the-art results on geo-localization benchmarks: Im2GPS3k, YFCC26k, GWS15k, and the Geo-Tagged NUS-Wide Dataset.
- The location encoder can be used as a pre-trained GPS encoder to assist other geo-aware neural architectures.
- Provides a pre-trained location encoder that can be used standalone for GPS embedding tasks, such as GPS-only multi-class classification.
Use cases
- Worldwide image geo-localization, predicting the GPS coordinates of a given image.
- Generating GPS embeddings that can be combined with visual features for geo-aware image classification.
- GPS-only multi-class classification, where location features alone are used for classification tasks.
Getting started
Install via pip:pip install geoclipOr install from source:git clone https://github.com/VicenteVivan/geo-clipcd geo-clippython -m pip install .For inference and usage examples, see the repository README.
License and pricing
Licensed under the MIT License. This is an open-source project.
Based on the project's public README and repository metadata.