Using a modular approach, this algorithm supports true multi-modal pipelining where different modes of data can be independently processed and combined to produce a knowledge-rich, intercorrelated database that can abstractify complex situational data. This allows for constant expansion of new sensors and data streams that readily integrate with existing ones allowing for a dynamic, ever-evolving data structure.