Semantically Complex Query Image Retrieval using Scene Graph

Understanding semantically complex queries which include attributes of objects, actions, multiple objects with relationships between objects, or interactions remain a challenging task in image retrieval system. In this work, we propose a solution to this problem. Leveraging the state-of-the- art semantic query parser and detailed image captioning in the form of the scene graph, we design and implement a solution for the image retrieval task. We confirm the effectiveness of our proposed model by evaluating on Visual Genome image dataset.

Project Report