As leaders during the meteoric rise of radio giant SiriusXM, Dan Benjamin and Marlon Osbourne (DBMO) had front-row seats to the digital broadcasting revolution (and the content explosion it sparked). But while platforms for pushing out videos, podcasts, and blogs had proliferated and evolved, tools for keeping track of all that information had fallen behind – to the great detriment of creators and audiences.

Dan and Marlon believed an AI solution could deliver this critical information, and they asked us to help bring their vision to life.

To create one tool capable of analyzing various media types, our first priority was converting them to a common denominator: text. Written language was already the primary information vehicle for blogs and news outlets, and videos with audio and podcasts could be automatically converted into time-stamped transcripts that captured most of their meaning. Homogenizing the content’s format freed us to refine an AI application designed to pull relevant information out of text-based files (and to deliver its findings in an actionable, versatile format).

Since auto-transcription captures only the audio portion of multimedia content, the DBMO product also needed visual interpretation abilities to provide high quality video analysis. We incorporated image recognition features able to supplement voice transcriptions to provide a more complete catalog of where entities, themes, and topics are present throughout a piece of video content.

We combined the text/transcript and image data streams and used AI analyses to identify entities (location names, people), topics (events, news items), themes (media genre). Using similarity matching via Doc2Vec and other text-focused AI recognition and evaluation techniques, we trained DBMO to generate time-associated metadata ready to be leveraged for search engine and content optimization.