Multi-Task Scale Aware Continuous and Localizable Embeddings

NGA uses deep networks for many tasks including image registration, land cover segmentation, and object detection. Current deep learning approaches develop specialist networks for each task and type of data. Not only is this inefficient, because networks can’t be reused across tasks, this approach ignores correlations between tasks and data sources that can improve performance. In response, we will develop MultiSCALE, a scale-aware, task-conditioned network designed to operate across multiple, mutually supporting tasks. Multi-image tasks are typically performed by matching the ground sample distance (GSD) of the images, usually through information destroying down-sampling. Instead, MultiSCALE conditions its feature extraction on metadata, such as GSD, viewing angle, solar angle etc. generating features that model these differences. Crucially, rather than train a feature space that is invariant (blind to) to these differences, MultiSCALE’s metadata conditioning makes the network aware of differences, such as absolute size. Cascaded task conditioning will improve performance, for example detecting objects while using land cover to guide the search. MultiSCALE will create a general network, trained to perform many tasks across varying resolution data at their native resolutions, increasing network re-usability and improving task performance to support NGA’s imagery analysis.