As more and more media content - like video games, videos, films and cartoons - is created, it gets harder for production companies and investors to decide which content is worth promoting and investing capital into. There is a strong incentive to heavily filter through the content from creators on the basis of potential commercial success. While potential success is difficult to measure and is often determined based on personal experience, it can be formalised to predict how popular the piece of media in question will be with consumers. By analysing different aspects of already popular media, one can learn what determines the success of said media with a high degree of confidence.
Our client is a production company which helps creators promote media content, like mobile and browser video games, attract more customers and generally increase its popularity. Our client helps hundreds of independent creators and media development studios, working with dozens of new projects daily, providing consultations and giving recommendations on how their products can be improved.
In order to give more accurate recommendations and determine which projects will be successful in the future, our client asked us to develop a system for analysis of digital media, like a video game recording. There are multiple features which could be analysed, like dominant colors, camera angle, how fast the camera moves, etc. Such a system would provide invaluable insight about media that is already successful, like which color pallete is the most pleasing to consumers, but also help give recommendations to developers on how their products can be improved to gain more success on the market.
The application needed to determine the dominant colors, the camera angle, and camera speed by analysing a video recording of a video game. There were multiple challenges in the development of this application:
We first determined the framerate which would yield satisfactory results while maintaining quick processing speed. The app takes every 30th frame for analysis as it is minimal viable value in terms of accuracy and performance.
Each frame is being quantized by 16 colors, meaning all colors are divided into 16 groups, which is the most optimal in terms of processing time and visual perception. We have used OpenCV to quantize each video frame and generate a specter image. After all specter images are generated, they are quantized again to get an average for all frames.
Scene changes are best done via monitoring the color histogram and detecting sudden and drastic changes in dominant colors or a sudden appearance of one or more new colors. When such a change is detected, the application determines that a scene was changed and the color analysis process starts over for the new scene.
Determining the speed of camera movement is a sophisticated task due to a large number of objects moving at different speeds and in different directions within the frame. We have decided to use a Python library which is capable of determining the camera speed by locating static objects and building a motion vector. While the out-of-the-box solution worked well enough, there still were several problems, like the possible absence of static objects in the video or too many objects which move in a chaotic manner. We ended up augmenting the solution by developing custom object speed detection modules for more accurate results.
After the dominant colors are determined, the application compares the color specters to popular color harmonies to provide additional information for our client. Depending on how much the media in question adheres to a certain color harmony, recommendations on possible design changes can be given based on information on previous projects.
The application is currently used as part of a recommendation system for our client. It helped to analyse previous projects and detect what makes a successful media product and how new products can be improved. For us as a team this project was another interesting dive into computer vision.